TLDR AI · DEEP DIVES & ANALYSIS

LLMS ARE GETTING BETTER AT CHARACTER-LEVEL TEXT MANIPULATION

blog.burkert.me7 min read

Summary from the TLDR AI newsletter

Newer models are becoming more adept at manipulating text at the character level, despite their understanding of text being based on tokens. Most current-generation models are able to decode Base64 text. They are now able to substitute characters fairly reliably. Character-level operations are far from solved, but it is clear they have made huge progress in the area.