XMODEL-LM

Summary from the TLDR AI newsletter

Xmodel-LM is a compact and efficient 1.1B language model pre-trained on around 2 trillion tokens. Trained on a self-built dataset (Xdata), which balances Chinese and English corpora based on downstream task optimization, Xmodel-LM exhibits remarkable performance despite its smaller size. It notably surpasses existing open-source language models of similar scale.