REPO STATE LOOPHOLES DURING AGENTIC EVALUATION

Summary from the TLDR Web Dev newsletter

SWE-bench (the popular AI benchmark for SWE tasks) has identified that AI agents can exploit loopholes by accessing future repository states (including solutions) through `git` commands, leading to inaccurate benchmark results.