CHEAPER VLM TRAINING

Summary from the TLDR AI newsletter

Meta researchers introduced a method called zero-shot grafting, which trains a vision encoder with a smaller surrogate model derived from a large LLM's shallow layers. This reduces VLM training costs by ~45% while preserving or even improving performance when transferred to the full LLM.