EXTREME VIDEO COMPRESSION WITH PRE-TRAINED DIFFUSION MODELS

Summary from the TLDR AI newsletter

Diffusion models can be repurposed for their broad ‘knowledge’ of the world as they get better at synthesizing images and videos. This paper found a phenomenal 0.02 bits per pixel compression. The key trick here was to measure perceptual similarity along the way and resend an original video frame as needed.