DISTRIBUTED TRAINING OF DEEP LEARNING MODELS

Summary from the TLDR AI newsletter

A great introduction to the challenges and some of the algorithms behind distributed training of many modern deep learning models, this post discusses the challenges and bottlenecks of training models that are too large for a single GPU.