WHY DO OPENAI'S GPT-2 WEIGHTS BEAT MINE?

Summary from the TLDR AI newsletter

This researcher trained their own LLM from scratch but discovered that their models were worse at instruction-following than the original OpenAI GPT-2 small weights, even when their model got better results in a more technical evaluation. This post is the first in a series where the researcher looks at the nature of the problem and explores options for what might be the cause.