PRETRAINING OF LARGE IMAGE MODELS

Summary from the TLDR AI newsletter

Apple has released a great paper on pretraining image models with an autoregressive loss. It measures scaling behavior and finds that like in language, these large unsupervised image models have very predictable behavior.