TRITON IMPLEMENTATION OF NAIVE SPARSE ATTENTION

Summary from the TLDR AI newsletter

The DeepSeek NSA paper made waves last week for its scalable and efficient long context attention algorithm. However, there was no code available. This work is a Triton replication that can be slotted into any PyTorch codebase.