OLMO: OPEN LANGUAGE MODEL

Summary from the TLDR AI newsletter

AI2 released a fully open language model training framework. The models come with comprehensive resources including full training data, model weights, training and evaluation code, over 500 checkpoints for each model, and future plans for fine-tuning code, all released under the Apache 2.0 License. These resources are provided by AI2’s Dolma and WIMBD. They include evaluation frameworks Catwalk and Paloma and support for analysis and fine-tuning through Weights & Biases logs and HuggingFace revisions. The models are competitive with the best in class.