The Daily Index — Tuesday, November 21, 2023 · TLDR Index

PROGRAMMING, DESIGN & DATA SCIENCE

TLDR

TLDR

A GUIDE TO LLM INFERENCE AND PERFORMANCE

Using the full power of the GPU during LLM inference requires knowing if the inference is compute-bound or memory-bound. Calculating the operations per byte possible on a given GPU and comparing it to the arithmetic of a model's attention layers reveals where the bottleneck is. This information can be used to pick an appropriate GPU for model inference and use techniques like batching to better utilize GPU resources.

baseten.co · 18 min

TLDR

OPENAI’S MISALIGNMENT AND MICROSOFT’S GAIN

OpenAI fired Sam Altman on Friday. Altman was quickly hired by Microsoft. This has caused a talent drain within OpenAI, with a good portion of it going to Microsoft. The situation highlights the problems of running a non-profit organization. This article looks at OpenAI's non-profit model, the issues within OpenAI, Microsoft's involvement, and how these recent events will change the AI landscape.

stratechery.com · 20 min

TLDR

A SECRET GOOGLE DEAL LET SPOTIFY COMPLETELY BYPASS ANDROID’S APP STORE FEES

Spotify doesn't pay any commission fees when it uses its own payment processor on Android. It only pays a 4% fee when users choose to use Google as their payment processor. Google fought to keep the terms of its deal with Spotify secret during its antitrust trial as it could damage negotiations with other app developers who might want more generous fees. Spotify's unprecedented popularity was enough to justify a bespoke deal.

theverge.com · 2 min

TLDR

CRUISE CO-FOUNDER RESIGNS FOLLOWING CEO EXIT

Cruise's co-founder and chief product officer Daniel Kan has resigned just a day after the company's CEO Kyle Vogt announced his resignation.

engadget.com · 2 min

TLDR

WHO IS EMMETT SHEAR, OPENAI’S THIRD CEO IN THREE DAYS?

Emmett Shear is the co-founder of the video game live-streaming company Twitch.

cnn.com · 7 min

TLDR

YOUTUBE IS REPORTEDLY SLOWING DOWN VIDEOS FOR FIREFOX USERS

Some Firefox users are experiencing a five-second delay when loading YouTube videos.

androidauthority.com · 3 min

TLDR

THE COST OF INDEX EVERYTHING

More information isn't always better - index size is a trade-off against retrieval quality.

matt-rickard.com · 2 min

TLDR

SAM ALTMAN IS STILL TRYING TO RETURN AS OPENAI CEO

Multiple sources say that Sam Altman and Greg Brockman are willing to return to OpenAI if the remaining board members who fired Altman stepped aside.

theverge.com · 6 min