AN EVOLVED UNIVERSAL TRANSFORMER MEMORY

Summary from the TLDR AI newsletter

Sakana AI has developed a new transferable memory module that works by compressing attention information and transferring it between models. It slightly improves performance on some long context benchmarks.