VISION LANGUAGE MODELS ARE BIASED

Summary from the TLDR Web Dev newsletter

This paper shows that Vision Language Models show a lot of bias, relying on memorized knowledge rather than visual analysis, which leads to big failures when presented with slightly modified images.