LLM Tokens per Second: Decode Throughput (2026)

Decode throughput measures how many completion tokens a model streams each second once generation has begun, and in our 2026-07-26 streaming benchmark GPT-4o led every endpoint at 174 tok/s while DeepSeek V4-Flash native was the fastest Chinese endpoint at 95.8 tok/s.

Tokens per second is the metric people quote when they say a model "feels fast," but it answers only half the latency question. It tells you nothing about how long you wait before the first word appears. Below we separate the two, then rank seven endpoints on measured decode speed from a single-prompt streaming snapshot. All numbers are a 3-run median captured on 2026-07-26 from one network location. DeepSeek V4-Flash was measured on its native endpoint api.deepseek.com; every other figure was OpenRouter-routed and needs native re-verification.

What decode throughput is, and what it is not

Decode throughput is a streaming metric that counts the completion tokens a model emits per second after the first token has arrived, as distinct from time to first token (TTFT), which measures the wait before anything at all appears on screen. A model can win one and lose the other.

TTFT governs how responsive a chat interface feels on the first keystroke. Decode throughput governs how fast a long answer finishes rendering once it starts. For a 20-token confirmation reply, TTFT dominates the felt latency. For a 2,000-token report, code file, or batch summarization job, decode throughput dominates: at 40 tok/s that output takes about 50 seconds of streaming, at 174 tok/s about 11.

So the right question is not "which model is fastest" but "fastest at which stage, for which output length." A snappy TTFT with slow decode is great for autocomplete and terrible for long-form generation. The reverse is true for batch pipelines that do not care about the first token.

LLM tokens per second: decode throughput compared (verified 2026-07)

The table ranks median decode throughput across three streaming runs per endpoint. Reasoning-model rows also list the wait to their first content token, because that delay is separate from decode speed and can dwarf it.

GPT-4o wins decode throughput outright at 174 tok/s, so no Chinese endpoint in this set is the fastest on tokens per second overall. Among Chinese endpoints, DeepSeek V4-Flash native is fastest at 95.8 tok/s. The fastest routed Chinese decode is GLM-5 at 70.9 tok/s, but only after a 13.9-second wait for its first content token, followed by MiniMax-M2.5 (63.4) and Qwen3-235B (62.8).

First-hand: how we measured this

We streamed each of the seven endpoints three times on 2026-07-26 using one fixed prompt ("List 10 numbered one-sentence facts about the HTTP protocol"), temperature 0 and max_tokens 4000, over server-sent-event streaming, and reported the median of three runs. Decode throughput is completion tokens (reasoning tokens included) divided by the decode window, measured from the first streamed token of any kind to completion. On reasoning models that makes it a blended rate across thinking and content phases.

The headline decode medians we measured were GPT-4o at 174.0 tok/s, DeepSeek V4-Flash native at 95.8 tok/s, GLM-5 at 70.9 tok/s, MiniMax-M2.5 at 63.4, Qwen3-235B at 62.8, DeepSeek routed at 39.5, and Kimi K2 at 39.4. All 21 runs completed without error. This is a single-prompt, single-region snapshot, not a sustained load test, and routed figures still need native re-verification.

According to DeepSeek's API documentation, the native deepseek-chat model maps to its latest V-series build served from api.deepseek.com, which is the endpoint that produced the 95.8 tok/s and 0.71s TTFT figures here.

Why routing roughly halved DeepSeek decode

We measured the same DeepSeek family twice at the same moment: once natively and once through an aggregator. The native path streamed at 95.8 tok/s with a 0.71s TTFT; the routed path streamed at 39.5 tok/s with a 2.54s TTFT. Routing added about 1.8 seconds to the first token and roughly halved decode throughput in our runs.

Treat this as a routing-overhead signal, not a perfect controlled comparison. The routed deepseek/deepseek-chat alias may point at a different DeepSeek build than the native V4-Flash, and the routed request may land on a different physical host. The direction of the effect is clear even if the exact multiple is not: an extra network hop and a different backend cost you both first-token latency and steady-state throughput.

Reasoning models and the throughput mirage

GLM-5 and MiniMax-M2.5 behaved as reasoning models on this prompt. They streamed thinking tokens almost immediately (TTFT 2.50s and 1.50s), but the first token a user would actually read arrived at 13.89s and 10.68s respectively, after 807 and 449 reasoning tokens. Their blended decode rates across thinking plus content (70.9 and 63.4 tok/s) look competitive, yet the total wait to a finished answer stretched to 18.61s and 12.46s.

This is a fit question, not a quality knock. Frame it as a trade-off: the thinking pass is a latency tax on simple generation like the list prompt here, and it may be well spent on hard multi-step problems. According to MiniMax's platform documentation, the M-series models are designed to emit an internal reasoning trace before the final answer, which is exactly the behavior we observed. If your workload is short and interactive, that reasoning tax is pure overhead; if it is genuinely hard, the extra seconds may buy a better result. This cluster measured speed only, so we make no accuracy claim either way.

What actually drives tokens per second

Decode throughput is not a fixed model property. It moves with the serving stack, batch pressure, output length, and your distance from the region hosting the model. The same weights on a busy shared endpoint at peak load will stream slower than on a quiet dedicated one.

According to OpenAI's API documentation, streaming responses return tokens as server-sent events as they are generated, so the throughput you observe reflects generation speed under current load rather than a guaranteed rate. That is why a single-region, single-prompt snapshot like ours is directional, not a service-level guarantee. Cheap and fast are also different axes: Qwen3-235B posted the slowest TTFT in this set at 3.16s despite being the cheapest option in our earlier task benchmark, so a low per-token price does not imply high tokens per second.

FAQ

Is tokens per second the same as TTFT? No. Tokens per second is decode throughput, the streaming rate after generation starts. TTFT is the wait before the first token appears. A model can lead on one and trail on the other, so quote both when you compare latency.

Which endpoint had the fastest decode throughput? GPT-4o, the Western baseline, at a median 174 tok/s across three runs on 2026-07-26. Among Chinese endpoints, DeepSeek V4-Flash on its native api.deepseek.com endpoint was fastest at 95.8 tok/s. All routed figures still need native re-verification.

Why did GLM-5 show a high decode rate but slow total time? GLM-5 took about 13.9 seconds from request start to its first content token, most of it spent emitting reasoning tokens. Its blended decode rate across thinking and content was 70.9 tok/s, but the thinking delay pushed total wall time to 18.61s. Decode speed and time-to-answer are separate measurements.

Does a lower price mean higher tokens per second? No. Price and throughput are independent axes. Qwen3-235B was cheapest in our earlier 2026-07-10 task benchmark yet posted the slowest TTFT here at 3.16s. A cheap per-token rate tells you nothing about how fast tokens stream.

Are these numbers a guaranteed speed? No. They are a 3-run median from one prompt, one region, and one moment on 2026-07-26. Real throughput shifts with load, output length, region, and serving stack. Use the ranking as a directional signal, not a service-level guarantee.

For the full cluster methodology and every latency metric side by side, see our hub on the fastest Chinese LLM API, or match a model to a workload in our guide to the best Chinese LLM by use case.

Last verified: 2026-07-26. Author: Kevin Fan, Customer Success Manager, who benchmarks Chinese LLM APIs for buyers comparing providers before sign-up.