TLDR AI · ENGINEERING & RESOURCES
STAR ATTENTION
Summary from the TLDR AI newsletter
Up to 11x speed improvement for long context token processing and generation. It is slightly better than ring attention, but still worse than full attention.
TLDR AI · ENGINEERING & RESOURCES
Up to 11x speed improvement for long context token processing and generation. It is slightly better than ring attention, but still worse than full attention.