MRT5: DYNAMIC TOKEN MERGING FOR EFFICIENT BYTE-LEVEL LANGUAGE MODELS

Summary from the TLDR AI newsletter

Byte Level language models are a step towards a token free future. However, the sequence length issue is a huge challenge. Merge tokens dynamically to improve the number of tokens in context.