LIVECODEBENCH: HOLISTIC AND CONTAMINATION FREE EVALUATION OF LARGE LANGUAGE MODELS FOR CODE

Summary from the TLDR AI newsletter

Evaluating language models trained to code is a challenging task. Most folks use HumanEval from OpenAI. However, some open models seem to overfit to this benchmark. LiveCodeBench is a way to measure coding performance while mitigating contamination concerns.