ONE-MINUTE VIDEO GENERATION WITH TEST-TIME TRAINING

Summary from the TLDR AI newsletter

This work introduces Test-Time Training (TTT) layers, featuring highly expressive hidden states, to overcome the limitations of standard Transformers and alternatives like Mamba in generating long, coherent videos. By integrating TTT layers into a pre-trained model, they demonstrate the generation of coherent one-minute videos from text storyboards that significantly outperform baseline methods in telling complex stories according to human evaluations. Tom and Jerry cartoons are the test bed.