FINEWEB: 15T HIGH QUALITY WEB TOKENS (HUGGINGFACE HUB)

Summary from the TLDR AI newsletter

The newest Llama 3 models were trained on 15T tokens. This new dataset contains an extensively deduplicated corpus from common crawl and results in high quality models.