COMPILEBENCH: CAN AI COMPILE 22-YEAR-OLD CODE?

Summary from the TLDR Web Dev newsletter

CompileBench is a new benchmark created to test the ability of LLMs to handle complex software development tasks like dependency management and legacy code compilation. The benchmark tested 19 LLMs on 15 real-world tasks, including cross-compiling and resurrecting old code, showing varying levels of success and identifying issues like cheating. Anthropic models performed best in overall success and speed, while OpenAI models were the most cost-efficient.