LLMLINGUA

Summary from the TLDR AI newsletter

LLMLingua uses a well-trained small language model after alignment to detect unimportant tokens in a prompt and enable inference with the compressed prompt in black-box LLMs, achieving up to 20x compression with minimal performance loss.