WE'RE ACTUALLY RUNNING OUT OF BENCHMARKS TO UPPER BOUND AI CAPABILITIES

Summary from the TLDR AI newsletter

METR's Time Horizon suite is being saturated. Frontier AI models can reliably do all but maybe a dozen or so tasks in the suite, making it hard to upper bound their time horizon. New benchmarks are becoming more expensive to grade and create. The situation will likely get worse as AI progress continues. It is likely that, by mid-2027, no benchmark score from a 2026 or earlier benchmark will be able to rule out dangerous capabilities from frontier AI systems.