INSTRUCTION TUNING LARGE CODE MODELS

Summary from the TLDR AI newsletter

Most code models are instruction tuned on GPT-4 distillation data. This new work shows how to avoid that, introduces an extended human eval dataset, and gets 46.7 on the original human eval. While this is impressive, it showcases the extreme gap between closed and open code models.