EFFICIENT TRANSFORMER WITH KEY-VALUE SHARING

Summary from the TLDR AI newsletter

Multi-Layer Key-Value (MLKV) sharing is a new method that significantly reduces memory usage during auto-regressive inference in transformers.