PREDICTING WHEN RL TRAINING BREAKS CHAIN-OF-THOUGHT MONITORABILITY

Summary from the TLDR AI newsletter

Researchers propose a framework that predicts when RL training degrades Chain-of-Thought (CoT) monitorability by examining reward conflicts. They categorize rewards as "In-Conflict," "Orthogonal," or "Aligned," predicting their impact on CoT transparency. Empirical tests confirm the framework's predictive accuracy, showing "In-Conflict" rewards reduce transparency, whereas "Orthogonal" and "Aligned" rewards maintain it.