TLDR · QUICK LINKS
COMPRESSION IS PREDICTION
Summary from the TLDR newsletter
In compression, entropy measures how small we can shrink things, but in language modeling, it's used to make models better at prediction.
TLDR · QUICK LINKS
In compression, entropy measures how small we can shrink things, but in language modeling, it's used to make models better at prediction.