MODERNBERT

Summary from the TLDR AI newsletter

BERT is an encoder only style model that is one of the most heavily used language models on the market, used in Google search for essentially every query. This new model from Answer AI, LightOn, and friends is faster, modern, and extremely performant. It can be used as a drop in replacement. They use a number of modern tricks like batch ramp to improve overall model performance.