PRETRAINING DATA, NOT VERIFIABILITY, IS WHY LLMS ARE ESPECIALLY GOOD AT MATH (AND CODING)

Summary from the TLDR AI newsletter

Large language models are good at judging math arguments because they are still mostly powered by imitative learning rather than reinforcement learning. LLMs are especially good at math because almost everything in the math literature is correct. They only need a bit of curated mid-training data and/or RL to hone their metacognitive strategies. In many other fields, the research literature is a bit of a dumpster fire, with some true and valuable information mixed into a sea of falsehoods and confused ideas. This causes LLMs to spit out tons of confused nonsense with occasional insights.