THE ERA OF EXPLORATION

Summary from the TLDR Web Dev newsletter

Future AI progress will depend more on exploration, acquiring new and informative experiences, rather than just scaling parameters. Pretraining LLMs inadvertently pays an "exploration tax" by learning a rich sampling distribution, allowing smaller models to later distill reasoning abilities. Exploration has two key axes: "world sampling," deciding which problems to learn from, and "path sampling," determining how to gather data within those problems.