MULTI-STEP PREFERENCE OPTIMIZATION FOR RLHF

Summary from the TLDR AI newsletter

This paper introduces Multi-step Preference Optimization (MPO), a reinforcement learning approach that models alignment as a two-player Markov game to improve multi-turn interactions.