LONG-CONTEXT HANDLING FOR LLMS

Summary from the TLDR AI newsletter

DuoAttention is a framework designed to reduce memory and latency in long-context LLMs by selectively applying full KV caching to only essential attention heads.