OPEN CODE MODEL BEATS CHATGPT AT CODING TASKS

Summary from the TLDR AI newsletter

Closed models are strong because they are actually code models disguised as language models. A lot of the open language models fall short because they don't gain reasoning skills from massive code pretraining. Here, we have Starcoder models instruction tuned on Evol instructions and achieving an astonishing 57 on the HumanEval Benchmarks. Models and code are available.