PYTORCH EXPERIMENTAL FLOAT 8 TRAINING

Summary from the TLDR AI newsletter

Training models with lower precision is faster and cheaper, but unstable. There has been lots of great recent research on quantized training. This repository builds on those foundations with easy-to-read and hackable code to enable float8 training.