HOW MUCH DO LANGUAGE MODELS MEMORIZE?

Summary from the TLDR AI newsletter

Researchers developed a method to separate true memorization from generalization by training models on random data where generalization is impossible versus real text. They discovered that models memorize training data until reaching capacity, then shift to learning general patterns, with GPT-style transformers storing ~3.6 bits of information per parameter. This explains why attempts to extract specific training data from modern LLMs often fail - their training datasets are much larger than their memorization capacity.