2X SPEED UP HF INFERENCE WITH STATIC KV CACHE (GITHUB GIST)

Summary from the TLDR AI newsletter

Speeding up inference can open up use cases. This code suggests a way to speed up inference with Llama models on Hugging Face.