KIMI LINEAR TECH REPORT HAS DROPPED! 🚀

Summary from the TLDR AI newsletter

Kimi Linear is a novel architecture that outperforms full attention with faster speeds and better performance. It offers up to a 75% reduction in KV cache usage and up to 6x decoding throughput at a 1M context length. Kimi Linear's open-sourced KDA kernels can be used as a drop-in replacement for full attention. The two models available were trained with 5.7T tokens.