DBRX MOE

Summary from the TLDR AI newsletter

Databrix and Mosaic have trained a 132B parameter MoE model with impressive performance. They trained the model on 3,000 H100s and have released the weights. The model is also available on the Databricks API.