MINIMAX SPARSE ATTENTION FOR MILLION-TOKEN CONTEXTS

Summary from the TLDR AI newsletter

MiniMax Sparse Attention is a sparse attention architecture that uses group-specific Top-k block selection to scale long-context inference while preserving model quality. On a 109B multimodal model, it matched GQA performance while cutting attention compute by ~30x at 1M tokens.