EXPENSIVELY QUADRATIC: THE LLM AGENT COST CURVE

Summary from the TLDR AI newsletter

It doesn't take much to start spending a significant amount of tokens on context. This problem compounds as conversations get longer and only gets exponentially more important with agent and subagent workflows. Agent developers need to keep this problem in mind when developing AI tools. This post provides an interactive tool that shows how quickly context size can take up a large percentage of a cache read.