MINI-OMNI2: TOWARDS OPEN-SOURCE GPT-4O WITH VISION, SPEECH, AND DUPLEX CAPABILITIES

Summary from the TLDR AI newsletter

A powerful and open source attempt to recreate GPT-4o speech capabilities. This particular model was trained by aligning modalities with pre trained audio and speech encoders.