Daily Edition · Page 3

THE DAILY INDEX

Technology, AI, software, security, crypto and business

HEADLINES & LAUNCHES

TLDR AI

NVIDIA'S ACHILLES HEEL: INFERENCE

Benchmark's Eric Vishria shares his email to investors about Nvidia from 2024. Nvidia's dominance holds in training, but GenAI inference is emerging as the weak spot. Purpose-built SRAM architectures from players like Groq and Cerebras deliver order-of-magnitude faster inference than GPUs, unlocking larger models, lower latency, and new use cases. As inference becomes a growing share of spend, performance gaps could shift AI compute winners away from incumbents.

ON NEURAL SCALING AND THE QUANTA HYPOTHESIS

Humanity started an experiment in scaling up deep neural networks several years ago. The experiment is on track to be the most expensive ever attempted, and we don't yet know what the results of the experiment will be. Despite its importance, there isn't a mature theory backing the experiment. We don't really know how to think about what neural networks are doing internally. The number of people working on such a theory in public is relatively small.

AI IS EVERYWHERE, BUT NOWHERE IN RECENT PRODUCTIVITY DATA

Forrester's J.P. Gownder argues that AI hasn't yet boosted productivity, similar to how PCs didn't immediately impact productivity metrics. His research suggests AI could replace 6% of jobs by 2030, with many companies yet to see tangible ROI from AI projects. He contrasts AI's slow job-replacement with previous shifts like outsourcing, highlighting financial decisions rather than direct AI replacements driving current job losses.

TLDR IndexPage 3