Prompt caching explained: stop paying twice for the same context

The problem caching solves

Say you've built an assistant that answers questions about your product catalogue. Every question sends the same five-page catalogue plus your instructions, then a one-line question. Without caching, you pay full price for the catalogue every single time.

With caching, the provider stores the repeated start of the request for a short time. Later requests that start with exactly the same text read it from the cache at a much lower rate, and usually get an answer faster too.

How it works with the main providers

Anthropic, OpenAI and Google all offer prompt caching on their APIs, but the details differ. Some apply it automatically once a prompt is long enough, others need you to mark which part to cache. Most charge a little extra to write to the cache and much less to read from it, and the cache only lasts for a set time, from a few minutes to several hours, unless it's used again. At the time of writing, OpenAI caches automatically on prompts of 1,024 tokens or more, charges 1.25 times the input rate to write to its default 30-minute cache and up to 95 per cent less to read from it. Anthropic charges 1.25 times the input rate for a 5-minute cache or twice the rate for a 1-hour cache, and a cache read costs 2.5 to 10 per cent of the normal input price depending on the model. Google caches automatically on Gemini 2.5 and newer models once a prompt passes 2,048 to 4,096 tokens, charges 5 to 10 per cent of the input price for cached tokens, and adds an hourly storage fee if you create a cache yourself.

The key rule is the same everywhere: the cached part has to be identical and at the start of the request. Put fixed instructions and documents first and the changing question last.

When it's worth it

Caching pays off when a large, fixed block of text is sent many times in a short period: customer service assistants, document Q&A, coding tools and any workflow that runs through a batch of items with the same instructions.

It does little for one-off requests or prompts that change every time. In those cases, shorter prompts and a cheaper model will save more.

How to check it's working

The API response tells you how many tokens were read from the cache. Check this in your logs after setting it up. If the number is zero, something at the start of your prompt, such as a date or a user name, is changing each time and breaking the match.

How prompt caching works

When a large language model reads your prompt, it does a lot of work on every token before it starts writing an answer. Prompt caching stores the result of that work for the start of the prompt, often called the prompt prefix. Technically, the provider keeps the model's key value calculations in what's known as a KV cache, so it doesn't have to redo them.

The first request pays to write the prefix to the cache. Subsequent requests that begin with the same prefix get a cache hit and read the cached prefix instead of processing it again. If anything in the prefix is different, you get a cache miss and pay the normal rate for the entire prompt.

So the rule that matters is simple: the cached part must be identical and must come first. The changing part, such as the new user message, goes at the end.

What you can cache

Anything that stays the same across requests is a candidate: the system prompt and system instructions, tool definitions, long documents, product catalogues, policies and examples. In multi turn conversations, earlier conversation messages can be cached too, so each follow up request only pays full price for the newest part.

What breaks the cache

A cache miss usually means something near the start of the prompt text changed. Common culprits are a timestamp or today's date in the system prompt, a user's name placed before the instructions, tools added or removed between calls, or switching models mid conversation. Each model has its own cache, so changing model means starting again.

Claude prompt caching

Anthropic's Claude API lets you mark exactly what to cache. You add a cache control setting to a content block, and that point becomes a cache breakpoint. Everything up to the breakpoint is cached as one prefix. You can use explicit breakpoints at several points, for example after the tool definitions, after a long document and after the conversation history, so different parts can be reused separately.

Anthropic also offers automatic caching, which places the breakpoint for you on the last cacheable block and moves it forward as a conversation grows. That's the simplest way to enable prompt caching for chat style tools. There's a minimum cacheable prompt length that varies by model, so very short prompts won't be cached at all. At the time of writing, the minimum is 512 tokens on the current models (Claude Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5) and between 1,024 and 4,096 tokens on older ones. The cache lasts 5 minutes by default and resets for free each time it's read, or you can pay more for a 1-hour cache.

Claude Code, Anthropic's coding tool, relies heavily on prompt caching. Its instructions, tool definitions and project context stay the same through a session, so most of each new request is read from the cache. This is a good example of why caching matters so much for coding assistants and other tools with long, repeated context.

OpenAI prompt caching

OpenAI applies prompt caching automatically on supported models once a prompt passes a minimum length. You don't need to mark breakpoints. The system looks for the longest prefix it has already seen recently and reuses it, which is why the same structure rule applies: static content first, variable content last.

OpenAI also lets you pass a cache key with an API request so that similar requests are more likely to land on the same cache. On models before GPT-5.6 this helps cache hit rates when many users share the same instructions. On GPT-5.6 and later the key is optional and mainly keeps cache use separate for different customers. At the time of writing, GPT-5.6 and later models charge for cache writes: 1.25 times the normal input rate for the default 30-minute lifetime, or 2.5 times for a 12-hour lifetime in the Responses API. Cache reads cost 0.1 times the input rate, or 0.05 times on GPT-6.1 Sol and GPT-6.1 Terra. Older models have no write charge, and entries usually last 5 to 10 minutes without use, or up to 24 hours with extended retention.

Gemini context caching

Google calls it context caching and offers two kinds. Implicit caching works automatically on supported Gemini models: if your request shares a prefix with a recent one, you may get a discount on the cached part without doing anything.

Explicit caching lets you create a cache yourself, for example from a long document or video, and refer to it in later requests. You choose a cache TTL, which is how long it's kept, and you pay a storage charge for the time it exists as well as a reduced rate when it's read. That makes explicit caching useful for content you know you'll use many times, but it needs managing so you don't pay to store caches nobody uses. At the time of writing, the TTL defaults to one hour if you don't set one, with no minimum or maximum. Storage costs vary by model, for example $0.50 per million tokens per hour on Gemini 3.8 Flash until 31 December 2026, then $1. Gemini 4 Argon and later models don't support explicit caching and use implicit caching only.

What prompt caching costs

Pricing is set relative to the base input price of each model. In general, a cache write charge is a little higher than the base input price, and cache reads are much lower. Uncached input tokens, such as the new question at the end, are charged at the normal rate. Output tokens aren't affected by caching.

To work out the saving, split total input tokens into three parts: tokens written to the cache, tokens read from the cache and uncached tokens. Multiply each by its rate, which for each provider is its cache write rate, its cache read rate and its base input rate per million tokens. Compare that with what the same requests would cost without caching.

A simple worked example

Say your support assistant sends a 10,000-token catalogue and instructions with every question, and answers 60 questions an hour. Without caching, every question pays the base input price for the whole catalogue. With caching, the first request pays the write charge, and every following request in the cache lifetime pays the much lower read rate for the catalogue plus normal price for the short question. The more requests that reuse the same cached content, the bigger the saving.

Building a caching strategy

Good caching is mostly about how you structure each prompt. These habits make the biggest difference.

• Put static content first: tool definitions, system prompt, then fixed documents
• Put anything that changes, such as dates, names and the user's question, after the cached part
• Keep the same tools available for a whole conversation rather than adding and removing them
• Stick to one model per conversation, because each model has its own cache
• Add new information as a new message rather than editing the instructions
• Group repetitive tasks so requests with the same prefix run close together

Cache lifetime and keeping a warm cache

Cache entries don't last for ever. Each provider keeps a cached prefix for a minimum lifetime, and many reset that timer each time the cache is read. If no request uses it within that time, the entry expires and the next request has to write it again. At the time of writing, Anthropic's cache lasts 5 minutes by default, or 1 hour at a higher write price. OpenAI's lasts at least 30 minutes by default on GPT-5.6 and later, or 12 hours at a higher write price, and usually 5 to 10 minutes without use on older models unless you turn on 24-hour retention. Gemini's explicit caches last for the TTL you set, one hour by default.

For busy tools, the cache stays warm on its own. For quieter ones, some providers offer a longer cache lifetime at a higher write price. Whether that's worth it depends on how often requests arrive. A cache that keeps expiring between requests can cost more than no cache at all, because you keep paying the write charge.

How to measure cache hit rates

Every provider reports caching in the API response. Look at the usage fields: they show how many tokens were written to the cache, how many were read from it and how many were processed normally. The names differ by provider, but the idea is the same.

Log these numbers for a few days and work out your cache hit rate: the share of input tokens read from the cache. If it's low, look at the start of your prompts for anything that changes between calls. Fixing one stray timestamp can turn a stream of cache misses into hits.

Prompt caching and your data

Caching stores a processed version of your prompt on the provider's systems for a short time. Anthropic and OpenAI both state that caches aren't shared between different customer organisations, but check each provider's data policy and your own UK GDPR position before caching documents that contain personal data. Anthropic also separates caches by workspace within an organisation on its own API. OpenAI says cached data held on its machines isn't kept beyond 24 hours. Google's Gemini caching docs don't cover this directly, so check its Gemini API terms.