HOW GOOD ARE FRONTIER MODELS AT PHYSICS? EXPERT RE-GRADING REVEALS BROKEN EVALUATIONS AND NEAR-SATURATION OF LEADING BENCHMARKS

Summary from the TLDR AI newsletter

Those scary physics scores were often the test's fault. Experts rechecked six popular benchmarks and found wrong answer keys, fuzzy questions, and grader bugs behind most model “fails.” Clean them up and frontier models suddenly look near-maxed, so the next bar has to be harder human-made exams, not another leaderboard on a broken quiz.