MMLU PRO

Summary from the TLDR AI newsletter

MMLU is a common benchmark for reasoning tasks. It is often considered to be both the gold standard and something models have overfit. MMLU Pro is a new, harder, and cleaner benchmark for measuring language model reasoning.