THERE IS NO DATA-GENERATING DISTRIBUTION

Summary from the TLDR AI newsletter

The data-generating distribution is a convenient mnemonic crutch for machine learning engineers. However, it isn't necessary for understanding machine learning. The same methods are used regardless of whether the world is producing randomness. We need to figure out when we actually need statistical models of data.