STATIC QUANTIZATION FOR LLMS

Summary from the TLDR AI newsletter

PrefixQuant is a new method that improves LLM quantization by isolating outlier tokens offline, eliminating the need for costly per-token dynamic quantization.