HIGH THROUGHPUT AND MEMORY EFFICIENT TRANSFORMERS

Summary from the TLDR AI newsletter

As Transformers have started to gain more and more traction, they are being served far more often than trained. This can be quite slow and expensive, but with some clever optimizations around memory layout on GPU, they get many multiples faster.