MEET LLMC: BEATING ALL COMPRESSION WITH LLMS

Summary from the TLDR AI newsletter

LLMc uses AI models to compress natural language. It serves as a high-capacity probabilistic reference system, enabling compression based on token prediction and rank encoding. It efficiently identifies high-likelihood token sequences and encodes them into a compact form using LLMs' in-context capabilities. Benchmarks show that LLMc outperforms traditional compressors such as ZIP and LZMA across a range of datasets.