MIRRORCODE

Summary from the TLDR AI newsletter

MirrorCode is a benchmark that tests AI models on long-horizon tasks. It contains tasks where AI models have to reimplement entire programs end-to-end without access to the original source code. AI-generated solutions must match the original program's output exactly on end-to-end tests. The benchmark's 25 target programs span different areas of computing, including Unix utilities, data serialization and query tools, bioinformatics, interpreters, static analysis, cryptography, and compression.