ROADMAPBENCH: EVALUATING LONG-HORIZON AGENTIC SOFTWARE DEVELOPMENT ACROSS VERSION UPGRADES

Summary from the TLDR AI newsletter

RoadmapBench evaluates long-horizon coding tasks, spanning multiple files and languages, grounded in real version upgrades across 17 repositories. The benchmark tests 115 tasks, requiring agents to implement functionalities with a median modification of 3,700 lines across 51 files.