TLDR AI · RESEARCH & INNOVATION
HYPERATTENTION: USED IN GEMINI
Summary from the TLDR AI newsletter
It is widely believed, and unofficially confirmed, that Gemini’s amazing 1m+ token context window uses HyperAttention to achieve such success.
TLDR AI · RESEARCH & INNOVATION
It is widely believed, and unofficially confirmed, that Gemini’s amazing 1m+ token context window uses HyperAttention to achieve such success.