Daily Edition · Page 3

THE DAILY INDEX

Technology, AI, software, security, crypto and business

GENERAL

TLDR AI

HOW PERSISTENT IS THE INFERENCE COST BURDEN?

Inference costs may not be that much of a bottleneck for AI progress. The cost to reach a given capability level falls fast, so the inference cost burden is more transient than it might appear from looking at only frontier models at launch. The data on RL scaling is still thin, so it is difficult to draw conclusions yet. It will be interesting to see how quickly cheaper models catch up to frontier capability levels, and how inference costs for fixed tasks decrease over time.

HOW MUCH ARE AI REASONING GAINS CONFOUNDED BY EXPANDING THE TRAINING CORPUS 10,000X?

Benchmark performance gives biased estimates of out-of-distribution generalization if LLM training data is polluted with benchmark test data. Typical decontamination filters fail to detect semantic duplicates. This suggests that recent benchmark gains are confounded - the prevalence of soft contamination means gains reflect both genuine compatibility improvements and the accumulation of test data and effective test data in the growing training corpora.

ANNOUNCING SPREADSHEET ARENA

Spreadsheet Arena is an open platform for evaluating LLM-generated spreadsheets. Formatting and structure often influence user preference more than formula complexity. There are significant differences in domain-specific preferences, with academic models suffering from heavy formatting and finance models benefiting from professional color coding. Crowd preferences often diverge from expert ratings, particularly in color coding and formatting.

MICRON IS SPENDING $200 BILLION TO BREAK THE AI MEMORY BOTTLENECK

Micron Technology, the largest American maker of memory chips, is rushing to add manufacturing capacity to break the memory bottleneck. The company is spending $50 billion to more than double the size of its 450-acre campus. It will build two new chip factories, the first of which is expected to start production of DRAM in mid-2027. Micron also recently broke ground on a $100 billion fab complex in New York, and it announced a $9.6 billion fab investment in Japan last year.

TLDR IndexPage 3