THE TWO MMLU SCORES: WHAT A BENCHMARK NAME DOES NOT FIX

Summary from the TLDR AI newsletter

Two builds with the same provider, model family, metric identifier, unit, and benchmark name can score differently. The difference is the runners, graders, and dataset splits. The shared mmlu label identifies a dataset family, not a full measurement procedure.