Daily Edition · Page 3

THE DAILY INDEX

Technology, AI, software, security, crypto and business

HEADLINES & LAUNCHES

TLDR AI

INSIDE THINKING MACHINES' INTERACTION MODELS

Thinking Machines is an AI research lab focused on human-AI collaboration. It believes that real work benefits from continuous collaboration, where the human clarifies, redirects, and gives feedback as a model goes along. This requires an interface that supports that instead of treating the human as someone who hands off a task and walks away. Thinking Machines' interaction models make interactivity a part of the model itself. The company plans to open a limited research preview in the coming months, with a wider release later this year.

POPPING THE GPU BUBBLE

AI models typically produce one token at a time - you can't compute the third token before you have the second. The GPU does most of the heavy lifting, but there is also some work that needs to be done by the CPU. GPU bubbles occur when the GPU sits idle in a loop waiting for the GPU to complete its job. This article looks at how to hide these bubbles using a technique called pipeline decoding, which involves starting the GPU work on the next token while the CPU is still finishing the last one.

TLDR IndexPage 3