OPEN REPLICATION OF STATE-OF-THE-ART VISION/LANGUAGE MODELS

Summary from the TLDR AI newsletter

HuggingFace has released IDEFICS, which is quite the acronym (Image-aware Decoder Enhanced à la Flamingo with Interleaved Cross-attentionS). IDEFICS is essentially a replication of Flamingo, which is a text and image model that interleaves the two and allows for visual question answering and reasoning. It is built on Llama v1 and has 8B and 70B variants.