TRAINING LANGUAGE MODELS ON LOW RESOURCE LANGUAGES

Summary from the TLDR AI newsletter

This paper introduces FinGPT, a language model trained on Finnish. The researchers who created it found that data repetition with some tricks produced extremely smooth loss curves. This may serve as an easy fix for underrepresented languages on the internet.