UNDERSTANDING GROKKING

Summary from the TLDR AI newsletter

The PAIR group at Google has released a lovely explainer that goes deep into the topic of Grokking. Grokking is the dynamic process a model goes through during training that may point to a shift from memorizing to understanding. It isn't well understood in general, but this is a lovely introduction that covers much of the groundwork for this strange phenomenon.