DEMO: DECOUPLED MOMENTUM OPTIMIZATION

Summary from the TLDR AI newsletter

2.5x faster, 100x less communication volume, and written by the original Adam author. This new optimizer boasts impressive performance for language model training with some strong improvements over the current dominant optimization schemes.