SCALING LAWS FOR DILOCO

Summary from the TLDR AI newsletter

DeepMind has released a paper detailing scaling laws for the powerful cross data center training algorithm DiLoCo. These show how stable model training can be even with gradients synchronized across continents.