DON'T TRUST LARGE CONTEXT WINDOWS

Summary from the TLDR Dev newsletter

LLMs have a reliable "smart zone" that typically degrades into a "dumb zone" once input exceeds approximately 100,000 tokens. Although developers market context windows reaching into the millions, performance studies show that effective attention is only a small fraction of those advertised figures.