DEEPMIND WORKING ON DISTRIBUTED TRAINING OF LARGE AI MODELS

Summary from the TLDR newsletter

DeepMind has published research that discusses how to distribute the training of models with billions of parameters among clusters of computers (that could theoretically be widely separated) while producing the same level of quality as before.