DEEPSPEED'S DEEPCOMPILE

Summary from the TLDR AI newsletter

The DeepSpeed team has worked to bring compilation to their distributed training efforts. This compilation speeds up various bottlenecked operations by many times. It makes use of a patched version of torch compile.