22580: FROM GPT2 TO KIMI3, EXPLAINED

Summary from the TLDR AI newsletter

KimiK3's gains come from more than scaling: it combines constant-state Kimi Delta Attention, periodic softmax retrieval, sparse experts, and selective residual access. Each architectural step improves how fixed-capacity memory stores, forgets, and retrieves information while preserving efficient inference.