HOW WELL DOES RL SCALE?

Summary from the TLDR AI newsletter

Improving AI capabilities either involves scaling the amount of compute used for RL during training or scaling the amount of compute used for inference during deployment. It's important to know where capability gains come from because scaling up the inference compute has very different implications than scaling up the training compute. While RL has provided impressive gains, we've reached a point where it is too expensive to go much further. This leaves inference-scaling as the remaining form of compute-scaling.