CHALLENGING MULTI-MODAL LANGUAGE MODELS WITH A NEW BENCHMARK

Summary from the TLDR AI newsletter

A new study reveals a weakness in multi-modal large language models (MLLMs) like GPT-4V: they struggle with specific types of image-text inputs, leading to errors. CorrelationQA is a benchmark designed to evaluate MLLMs' performance in scenarios where images may mislead or contradict text.