SWITCHHEAD: A MORE EFFICIENT TRANSFORMER MODEL

Summary from the TLDR AI newsletter

SwitchHead is a breakthrough in making AI models more efficient. It reduces the memory and computing needs of Transformers without losing performance.