3 TRILLION OPEN TOKEN DATASET RELEASED

Summary from the TLDR AI newsletter

There are many open datasets, but few of sufficient scale to train a frontier model. The Allen Institute for AI’s Dolma dataset aims to enable researchers to study data effects at scale.