PROMPT CACHING IN AGENTS

Summary from the TLDR AI newsletter

Prompt caching makes agents more economic, but this is quite fragile. A changed tool definition, a model switch, or a provider routing decision can turn a cheap incremental request into a full replay of the context. Cache behavior affects latency, cost, tool design, session design, and even which product features should be made available. This post looks at what the cache is, how it works, and common reasons for worse cache performance.