THE BITTER LESSON IS MISUNDERSTOOD

Summary from the TLDR AI newsletter

While general methods that leverage computation seem more effective, data still plays a very important role. You need 40% more data when doubling GPUs, otherwise you're just lighting cash on fire. AI companies have already consumed the internet. The leaders who win the next era of AI will be scaling data itself. This could be done by architecting better use of limited data or alchemizing entirely new ultimate data universes.