THERE ARE NO NEW IDEAS IN AI… ONLY NEW DATASETS

Summary from the TLDR AI newsletter

The next paradigm shift isn't going to come from an improvement to RL or a fancy new type of neural net. It will come when a new source of data that hasn't been accessed before is unlocked. One obvious source of information that a lot of people are working towards harnessing is video. Another source of information may be unlocked when scientists figure out how to gather and process information from cameras and sensors in a way that's amenable to training large models on GPUs.