JOHN SCHULMAN ON DEAD ENDS, SCALING RL, AND BUILDING RESEARCH INSTITUTIONS (51 MINUTE VIDEO)

Summary from the TLDR AI newsletter

John Schulman estimates that with full hindsight, a few talented people could have built a ChatGPT-3.5-level model in 2018-2019 with a couple of GPU boxes. He described early OpenAI as a "rag tag" blend of small exploratory research projects and bigger engineering efforts inspired by DeepMind's AlphaGo. He expects value functions and offline RL to make a comeback, and warns that catch-up mode makes it harder to build exploratory research culture later.