Daily Edition · Page 3

THE DAILY INDEX

Technology, AI, software, security, crypto and business

GENERAL

TLDR AI

IN SEARCH OF WASTED BITS: HOW MUCH INFORMATION DO LLM WEIGHTS CARRY?

A lot of LLM inference is transferring data from one place to another and then computing on it when it's there. The most frustrating bottleneck in the system is when compute units sit idle because the data bus feeding them isn't fast enough. The solution is to transform memory into compute. Quantization is a nice trick, but it doesn't actually trade memory for compute - it transfers half as much data to a place to do twice as much computation.

COMPUTER USE IS 45X MORE EXPENSIVE THAN STRUCTURED APIS

Vision agents are the default for operating web apps that don't expose APIs. Most teams default to vision agents because the alternative, writing an MCP or REST surface, is too expensive to build. The cost of the vision approach is treated as a fixed price. Current vision agents require detailed prompts to succeed in tasks, and they are still prone to making mistakes. Better vision models reduce error rates, but they do not reduce the number of screenshots required to reach the relevant data, each of which is worth thousands of input tokens.

ACCELERATING GEMMA 4: FASTER INFERENCE WITH MULTI-TOKEN PREDICTION DRAFTERS

Gemma 4 models reduce latency bottlenecks and achieve improved responsiveness for developers by using Multi-Token Prediction drafters. These drafters deliver up to a 3x speedup without any degradation in output quality or reasoning logic due to a specialized speculative decoding architecture. Speculative decoding decouples token generation from verification. It utilizes idle compute to 'predict' several future tokens at once with the drafter in less time than it takes for the target model to process just one token. The target model then verifies all of these suggested tokens in parallel.

GEMINI API FILE SEARCH IS NOW MULTIMODAL: BUILD EFFICIENT, VERIFIABLE RAG

Multimodal support, custom metadata filtering, and page-level citations are now available in the Gemini API File Search tool. The features can help developers bring structure to unstructured data for efficient, verifiable RAG. Users' RAG systems can now natively process and better organize text and visual data. The File Search tool handles the heavy infrastructure so users can focus on building products.

TLDR IndexPage 3