LLM Cache Hit Rate: What Actually Drives It (2026)

Cache hit rate is cached prompt tokens divided by total prompt tokens on a request. On a repeated 3,403-token prefix we measured 3,328 cached tokens on DeepSeek V4-Flash native, a 98% hit rate that cut computed input cost 88%.

Cache hit rate is a billing and latency metric that tells you what share of an incoming prompt the provider served from a stored prefix instead of reprocessing. It is not a quality metric. A 98% hit rate and a 0% hit rate produce the same model, the same weights and the same sampling behavior. What changes is the input line on the invoice and, usually, time to first token.

The formula every vendor's usage object supports:

hit rate = cached_tokens / prompt_tokens

DeepSeek exposes the split directly as prompt_cache_hit_tokens and prompt_cache_miss_tokens. Routed marketplace responses expose prompt_tokens_details.cached_tokens. Both reduce to the same ratio.

LLM cache hit rate (verified 2026-07)

We sent the same fixed ~1,800-token prefix, a fictional API changelog, two to three times per endpoint on 2026-07-28, two seconds apart, with a different one-line question appended each call, temperature 0 and max_tokens 40. Cache behavior was read from each API's own usage fields. All 17 calls succeeded and the routed portion billed $0.051 in total.

On DeepSeek V4-Flash called natively at api.deepseek.com, the first request reported 3,403 miss tokens and 0 hit tokens. The repeat request two seconds later reported 3,328 hit tokens against 3,403 prompt tokens, a 97.8% hit rate. Latency fell from 1.43s to 0.90s. A third call held at 3,328 hit tokens.

Here is the measured hit rate on the repeat call for every endpoint in the run. DeepSeek native was measured on api.deepseek.com; every other row was measured through a routed aggregator path, so those rows describe the route rather than the vendor's own API and need native re-verification.

Two readings of that table are wrong and worth killing early. The Qwen3-235B billed cost moved from $0.000726 to $0.000331 with 0 cached tokens reported, but that is not a measured cache: call B matches Qwen's list price on the routed marketplace while call A billed roughly twice list, so the delta tracks provider-price variance, not a hit. And the zeros for Claude, GPT-4o, Kimi and Qwen are a default-behavior finding from an implicit-only test, not evidence that those vendors lack caching. Each of them documents caching on its own platform.

The other honest split: routing did not uniformly destroy caching. DeepSeek's discount vanished through the router on this run while GLM-5 and MiniMax-M2.5 kept theirs, with the discount visible in billed cost. "Routers lose caching" is false as a blanket claim. It depends on the specific provider integration, on that day.

Why 98% and not 100%

The gap is arithmetic, not failure. 3,403 prompt tokens minus 3,328 hit tokens leaves 75 miss tokens, and 75 tokens is roughly the one-line question we changed on every call plus the small chat scaffolding around it. Prefix caching matches an exact prefix, so everything from the first differing token to the end of the prompt is a fresh miss.

That sets a practical ceiling. If your requests share a 3,000-token document and each appends a 200-token unique query, your best possible hit rate is about 94%, no matter how good the cache is. Hit rate is bounded by the ratio of shared prefix to total prompt, and the only lever that raises the ceiling is changing what sits where in the prompt.

What raises hit rate

Stable ordering is the whole game. A static system prompt and any long shared document placed first, with user-variable content appended last, produces the longest identical prefix across requests. Serialization matters as much as content: re-ordered JSON keys, a re-shuffled retrieved-document set or a changed whitespace convention all break the byte-identical match even when the meaning is unchanged.

Prefix length has to clear the vendor's floor. According to OpenAI's prompt-caching guide, caching is available for prompts of 1,024 tokens or more, and routing uses a hash of roughly the first 256 tokens of the prompt. According to Alibaba Cloud's Model Studio context-cache documentation, the implicit cache needs approximately 1,000 tokens to trigger on the qwen3.7-max series and 256 tokens on other models.

Request frequency is the third lever, because caches expire. Anthropic's documentation states a 5-minute default cache lifetime that refreshes at no extra cost each time the cached content is used, with a 1-hour option at twice the base input token price. Alibaba describes the implicit cache lifespan as indeterminate, cleared periodically as old data ages out.

What lowers it

According to Anthropic's prompt-caching documentation, prompts shorter than the model minimum, which is 1,024 tokens for Claude Sonnet 4.6, are processed without caching and no error is returned. That silence is the reason hit rate needs monitoring rather than assumption.

Reading hit rate from the API

DeepSeek is the cleanest observability case in this run because it separates the two halves of the ratio instead of reporting a single cached count. According to DeepSeek's API documentation, cache-hit input tokens bill at $0.014 per million against $0.14 per million for misses, and the discount applies automatically. Computed at those official rates, our measured split moved input cost from $0.000476 to $0.0000571 on the repeat call, an 88% reduction.

Those DeepSeek native dollar figures are computed from measured hit and miss tokens at published rates, not billed amounts, because the native API does not return a per-call cost. The routed figures quoted above are billed values from the marketplace usage object. Our full pricing breakdown lives in the DeepSeek cache discount analysis.

FAQ

What is a good LLM cache hit rate? For a workload with a genuinely stable prefix, above 90% is achievable and we measured 97.8% on DeepSeek native. For chat with short prompts and no shared document, near zero is normal and expected.

Does cache hit rate affect answer quality? No. Caching reuses stored prefix computation and affects input cost and latency only. Our run measured a latency drop from 1.43s to 0.90s on DeepSeek native alongside the cost reduction.

Why does my hit rate show 0% on an aggregator? Either the provider integration does not forward cache state, or the vendor requires explicit cache markers your client is not sending. We measured both patterns on 2026-07-28: DeepSeek routed at 0%, while GLM-5 and MiniMax-M2.5 routed above 97%.

Can I compute hit rate without vendor usage fields? Not reliably. Billed cost alone is misleading. Qwen3-235B's billed cost dropped 54% between our two calls with 0 cached tokens reported, which turned out to be marketplace price variance rather than a cache hit.

These figures are a single-region snapshot from 2026-07-28 and the routed rows still need native re-verification. For the full cross-vendor picture, including which caches survive routing and what they save, see our prompt caching benchmark hub.

Author: Kevin Fan, Customer Success Manager