CONTEXT ROT: HOW INCREASING INPUT TOKENS IMPACTS LLM PERFORMANCE

Summary from the TLDR AI newsletter

LLM performance degrades significantly as input length increases, even on simple tasks like text retrieval and replication. Multiple controlled experiments revealed that even frontier models don't process context uniformly, and performance becomes increasingly unreliable with longer inputs.