MODULAR NORM FOR NEURAL NETWORKS

Summary from the TLDR AI newsletter

Modular norm is a new method for normalizing weight updates in neural networks that scales training efficiently across different network sizes.