ROBUST FINE-TUNING OF ZERO SHOT MODELS

Summary from the TLDR AI newsletter

Zero shot models are AI models that have robust performance on a broad range of distributions. However, if you tune them on your narrow task, they tend to lose that ability while simultaneously getting much better at what they're trained for. This is good but could be improved. This paper presents a method to tune such models, like CLIP to maintain their generalizability while improving specific performance.