DECENTRALIZED DIFFUSION

Summary from the TLDR AI newsletter

Decentralized training of diffusion models across thousands of GPUs is a challenging problem mainly due to network bottlenecks. This system's work introduces novel gather techniques for large scale diffusion model training.