HOW WELL DOES RL SCALE?

Summary from the TLDR AI newsletter

RL-training for LLMs scales poorly. Most gains are from allowing LLMs to productively use longer chains of thought. This may be evidence that compute scaling will be less effective for AI progress than previously thought. The finding could lengthen timelines and affect strategies for AI governance and safety.