STEALING REASONING TRACES FROM PROPRIETARY LLM APIS

Summary from the TLDR AI newsletter

Proprietary reasoning can be recovered from its encrypted traces. Anthropic, OpenAI, and Google return encrypted chain-of-thought blocks to clients that can be replayed across sessions, users, and models. Researchers took a trace produced by a frontier model, replayed it into a weaker sibling, jailbreaked the weaker model, and recovered the stronger model's hidden reasoning in plaintext, without ever attacking the stronger model directly or triggering its anti-distillation safeguards. The decoded reasoning closely tracks the number of hidden thinking tokens reported by the API, and they contain real secrets and sensitive information.