TEACHING LANGUAGE MODELS TO LISTEN

Summary from the TLDR AI newsletter

This blog explores the current dominant paradigm for taking a pretrained language model and adding multimodal capabilities. Namely, aligning the embeddings between an encoder and the text model.