PROMPT CACHING: 10X CHEAPER LLM TOKENS, BUT HOW?

Summary from the TLDR AI newsletter

Catched input tokens are currently 10 times cheaper than regular input tokens for both OpenAI and Anthropic's APIs. They can even reduce latency by up to 85% for long prompts. It is clear that these providers aren't just saving responses and reusing them. This post takes a look at what's actually cached and how prompt caching works in general.