ON NEURAL SCALING AND THE QUANTA HYPOTHESIS

Summary from the TLDR AI newsletter

Humanity started an experiment in scaling up deep neural networks several years ago. The experiment is on track to be the most expensive ever attempted, and we don't yet know what the results of the experiment will be. Despite its importance, there isn't a mature theory backing the experiment. We don't really know how to think about what neural networks are doing internally. The number of people working on such a theory in public is relatively small.