THE RISE OF AI ‘REASONING' MODELS IS MAKING BENCHMARKING MORE EXPENSIVE

Summary from the TLDR newsletter

Reasoning models, which are supposed to be more capable than their non-reasoning counterparts in specific domains, are much more expensive to benchmark, making it difficult to independently verify claims.