MODELS GENERATING TRAINING DATA: HUGE WIN OR FAKE WIN?

Summary from the TLDR AI newsletter

There has been buzz the past few days around a number of works suggesting that models generating training data is a new unlock for performance. On the other hand, the community has pushed back, saying that the data processing inequality suggests this is impossible. What if the truth is somewhere in between and generating training data acts as a sort of filter for sampling from the distribution you actually care about?