HOW WELL MODELS UNDERSTAND SOUND AND VISION TOGETHER?

Summary from the TLDR AI newsletter

This study introduces AV-SUPERB, a new benchmark that tests how well a trained model understands both sound and visual data across a variety of tasks.