HOW MUCH ARE AI REASONING GAINS CONFOUNDED BY EXPANDING THE TRAINING CORPUS 10,000X?

Summary from the TLDR AI newsletter

Benchmark performance gives biased estimates of out-of-distribution generalization if LLM training data is polluted with benchmark test data. Typical decontamination filters fail to detect semantic duplicates. This suggests that recent benchmark gains are confounded - the prevalence of soft contamination means gains reflect both genuine compatibility improvements and the accumulation of test data and effective test data in the growing training corpora.