HOW CONFESSIONS CAN KEEP LANGUAGE MODELS HONEST

Summary from the TLDR AI newsletter

When models optimize for correctness, helpfulness, and safety all at once, they can learn shortcuts to “reward hack” while hiding misbehavior in plausible-looking outputs. OpenAI trained a version of GPT-5 to produce a separate "confession" judged only on honesty and found models will admit to reward hacking 90% of the time, even when they hide it in their primary response.