LEARNING TO SOLVE HARD PROBLEMS IN RL FOR LLMS BY NEVER GIVING UP

Summary from the TLDR AI newsletter

Standard RL results in disproportionately poor performance on the hardest problems. Simple scalar values may not be sufficient for accurate evaluations of large language models. This paper proposes a solution that reduces the compute spent on easy problems, which reallocates it towards harder problems. The solution appears to show substantial improvements on difficult tasks.