TLDR AI · DEEP DIVES & ANALYSIS

SOL LOVES TO CHEAT

jumploops.com14 min read

Summary from the TLDR AI newsletter

This developer tried to automate their dev flow with agents. The agents worked well, hitting 94% on Terminal Bench 2.1. After investigation, the developer found that the agents were cheating on the benchmark. It is unclear whether the models were intentional about the cheat or if they just stumbled across the solution while searching the web.