A VISION RESEARCHER'S GUIDE TO SOME RL STUFF: PPO & GRPO

Summary from the TLDR newsletter

This post presents a deep dive into Policy Optimization (PPO), one of the most popular algorithms used in RLHF for LLMs, and Group Relative Policy Optimization (GRPO), a method proposed by DeepSeek.