PYTORCH FLEXATTENTION

Summary from the TLDR AI newsletter

PyTorch has released a new API that allows any type of attention variant to be coded and compiled to Triton. This allows for better portability, performance, and research velocity on types of attention.