DIFFUSION MODELS WITHOUT ATTENTION

Summary from the TLDR AI newsletter

Most, but not all, modern diffusion models use the attention mechanism. The recent wave of interest in state spaces has made its way to diffusion, giving theoretical speed ups and opening up interesting applications.