What Is Prompt Caching? Explained With Live Numbers (2026)

Prompt caching is a provider-side optimization that stores the processed form of a prompt prefix so identical repeats skip recomputation, cutting input cost and time-to-first-token; our native DeepSeek test cut input cost 88% and latency from 1.43s to 0.90s.

Caching does not change what a model answers. It changes what the input side of the bill costs and how quickly the first token comes back. Everything below is anchored to one dated run rather than to a vendor claim.

What is prompt caching (verified 2026-07)

Prompt caching, also sold as context caching, is a server-side optimization that keeps the already-processed representation of a prompt prefix so a later request carrying that same prefix skips recomputation and bills those tokens at a reduced input rate. The unit of reuse is the prefix, not the prompt as a whole.

Matching starts at the first token and stops at the first byte that differs. That single property explains most of the behavior developers find surprising. Move one variable field to the top of your prompt and the cache stops firing for everything below it. Keep the stable material first and the reusable region grows with every turn.

According to DeepSeek API docs, cache-hit input tokens bill at $0.014 per million against $0.14 per million for a miss, and the discount applies automatically with no request flag. A 10x gap on the input side is large enough to reshape the unit economics of any workload that resends a long fixed context.

Implicit versus explicit cache designs

Two designs dominate, and the difference decides whether caching happens by default or only when you ask for it.

According to Anthropic's prompt caching documentation, Claude caching is driven by explicit cache_control breakpoints, where a cache write costs more than a normal input token and a cache read costs much less. Anthropic operates the Claude API, so its breakpoint model is the reference implementation of the explicit design.

OpenAI sits on the implicit side of that line on its own platform. According to OpenAI's prompt caching guide, prompts at or above 1,024 tokens are eligible for automatic caching with a discounted cached-input rate. Alibaba Cloud documents an implicit context cache for Qwen models on its native Model Studio endpoint, per Alibaba Cloud's context cache docs.

First-hand evidence: one prefix, sent three times

We sent the same fixed ~1,800-token prefix, a fictional API changelog, to DeepSeek V4-Flash on api.deepseek.com on 2026-07-28, three times, two seconds apart, changing only a one-line question at the end of each call. Temperature 0, max_tokens 40. Cache behavior was read from DeepSeek's own prompt_cache_hit_tokens and prompt_cache_miss_tokens fields.

On the repeat, 3,328 of 3,403 prompt tokens (98%) came back as cache hits. The input cost fell 88%, from $0.000476 to $0.0000571, computed from the measured hit/miss split at DeepSeek's official published rates rather than taken from a billing field, because the native endpoint does not return a per-call cost. Latency dropped from 1.43 s to 0.90 s, and the third call hit the same 3,328 tokens again.

The 75 missed tokens on call B are the tail that changed: the new question plus the assistant turn boundary. That is the shape of a healthy prefix cache. A near-total hit on the stable region, a small miss on whatever varies. Disclosure: DeepSeek was measured natively on api.deepseek.com, every other endpoint in this cluster was measured through OpenRouter, and the routed figures still need native re-verification.

What normally qualifies for a hit

The workloads that benefit share one trait: a large block of tokens that is byte-identical across calls and sits at the front of the request.

Long system prompts qualify. Retrieved document context in a RAG pipeline qualifies when the retrieval set is stable across a session. Few-shot example blocks qualify, and so does the accumulated history of an agent loop, since each turn replays everything before it. What does not qualify is a prompt that opens with a timestamp, a session ID, or a user name, because the mismatch at token one voids the whole prefix.

Vendor minimums and time-to-live windows differ, and they are the two settings most often misread. Check the doc for the endpoint you are actually billed against, not a summary. Moonshot AI runs a managed context cache with its own storage pricing on its native platform, documented at platform.moonshot.ai, which is a different commercial model again from a free automatic prefix cache.

Our test measured defaults, not capability

This run was designed to observe implicit caching only. It never sent an explicit cache marker, so any provider whose caching requires one was expected to report zero cached tokens. Claude Sonnet 4.6 and GPT-4o both showed 0 cached tokens and no billed change on the repeat through the routed path. For Anthropic that is exactly the documented design, not a capability gap. Kimi K2 also showed 0 cached tokens routed, while its native platform documents context caching.

The Qwen3-235B row deserves a specific caution, because its billed cost fell from $0.000726 to $0.000331 with 0 cached tokens reported. That is not cache evidence. Call B matches Qwen's list price on the routed marketplace at roughly $0.00032 computed, while call A billed about twice list, so the delta tracks provider-price variance, not caching.

Routing outcomes split rather than pointing one way. DeepSeek's cache was lost through the router on this run, 0 cached tokens and full price on both calls, while GLM-5 came back with 3,200 cached tokens and a 75% billed drop and MiniMax-M2.5 with 3,296 cached tokens and a 62% billed drop. The blanket claim that routers kill caching is false. Whether it survives depends on the specific provider integration, which we break down in the prompt caching benchmark hub. For the DeepSeek rate card itself, see our DeepSeek cache discount page.

FAQ

Does prompt caching change model output quality? No. Caching operates on the input side, reusing the processed prefix. It affects what you pay for input tokens and how fast the first token arrives, nothing else. We make no accuracy or quality claim from this run.

How do I know a cache hit actually happened? Read the usage object the API returns. DeepSeek native reports prompt_cache_hit_tokens and prompt_cache_miss_tokens directly. Routed responses reported prompt_tokens_details.cached_tokens plus a billed cost, which is how we caught the DeepSeek routed run showing zero.

Is caching free? It depends on the design. Automatic prefix caches such as DeepSeek's simply bill hit tokens at a lower rate. Explicit designs charge a premium for the cache write, and managed caches can carry separate storage pricing, so the break-even depends on how many reads follow each write.

Why did our test show no caching on Claude or GPT-4o? Because the test sent no explicit cache markers and used a routed path. Anthropic requires cache_control breakpoints that our design never sent, and OpenAI's automatic caching is documented on its own API rather than guaranteed through a third-party route.

Caching is the cheapest optimization available on a repeated-prefix workload, and the only reliable way to know whether you are getting it is to read the usage fields on your own traffic. Full provider-by-provider results, including the routed splits summarized above, are in the prompt caching benchmark hub.

Author: Kevin Fan, Customer Success Manager. Last verified: 2026-07-28.