RETHINKING THE DATA MOAT

Summary from the TLDR newsletter

Raw data from the internet is unlikely to be the best data distribution for AGI. Not all tokens are created equal. The scaling law can be improved by equalizing intelligence per token. Better quality tokens maximize the efficiency of compute.