HOW LARGE ARE LARGE LANGUAGE MODELS?

Summary from the TLDR Web Dev newsletter

This article goes over how LLMs' sizes have changed over time, starting with GPT-2 and GPT-3's parameter counts and training data specifics and ending with the limited public information about GPT-3.5 and GPT-4. Models like Mixtral and Deepseek V3 use a Mixture-of-Experts architecture, which makes them harder to compare to the traditional "dense" architectures.