HOW HAS DEEPSEEK IMPROVED THE TRANSFORMER ARCHITECTURE

Summary from the TLDR AI newsletter

DeepSeek is one of the only open model providers that seems to be innovating solidly on the underlying architecture of its generative models. Specifically, its research improvements are not the result of brute force, but instead come from minds that clearly understand what the Transformer does and operate in that paradigm.