NATIVE SPARSE ATTENTION: HARDWARE-ALIGNED AND NATIVELY TRAINABLE SPARSE ATTENTION

Summary from the TLDR AI newsletter

DeepSeek has entered the Attention Alternative arena with this novel algorithmic method to speed up quadratic Attention. They get as much as an 11x speed up with no loss in overall performance.