OPENAI'S AGI NUMBER CAME FROM A HARNESS, NOT THE MODEL

Summary from the TLDR AI newsletter

OpenAI claimed it had achieved AGI due to its 99.9% score on ARC-AGI-3. However, tests that ran the same model through the benchmark's own software scored 62.7%. The gap comes from the software around the model that OpenAI built. The different scaffolding around its agents helped OpenAI achieve the high score.