TRIATTENTION FOR KV CACHE COMPRESSION

Summary from the TLDR AI newsletter

TriAttention estimates KV importance in pre-RoPE space using stable Q/K centers and distance-based scoring, preserving long-context reasoning quality while sharply reducing KV memory use and improving throughput.