KRON OPTIMIZER

Summary from the TLDR AI newsletter

Kron is a new optimizer that is making the rounds as a strong alternative to second-order methods. It dramatically outperforms Adam on a number of baselines. This code is a drop-in optimizer for PyTorch.