From 390 KB to 890 Bytes

Abstract & How to Read This Paper

What this document is, who it is for, and how it is organized.

This paper answers one engineering question: how did DeepSeek cut the memory cost of a cached token from roughly 390,000 bytes to 890 bytes — a 438× reduction — in just under three years, and what exactly did each step cost and buy? The question matters because the KV cache is the quiet tax on everything a modern language model does. Every agent loop that re-feeds a 60,000-token transcript, every coding session that replays a repository, every reasoning chain that keeps its own scratchpad alive — all of it is paid for in cached keys and values. DeepSeek's own V4.1-Flash technical report opens by identifying this exact bottleneck: large KV caches strain HBM capacity, SSD capacity, and data-transfer bandwidth simultaneously, and together these demands constitute the primary obstacle to lowering deployment costs for long-horizon agents.

The history breaks into four discontinuities, and this paper treats each as a case study with the arithmetic shown in full. First, Multi-head Latent Attention (MLA), introduced in DeepSeek-V2 in May 2024: instead of caching per-head keys and values, cache a single 512-dimensional compressed latent plus a 64-dimensional rotary key per token, and reconstruct the rest on the fly. The official paper reports a 93.3% KV cache reduction against DeepSeek 67B while boosting maximum generation throughput 5.76×. Second, DeepSeek Sparse Attention (DSA), which arrived experimentally in V3.2-Exp in September 2025: a "lightning indexer" — 64 cheap multi-query heads scoring every cached token — selects the top 2,048 entries, and only those pay the full MLA bill, cutting the bytes read per decode step by roughly 5× at 131K context. Third, the hybrid compressed attention of DeepSeek-V4 in April 2026: Compressed Sparse Attention (CSA) pools four tokens into one cached entry while Heavily Compressed Attention (HCA) pools 128, together bringing a 1M-token context window within reach at 10% (V4-Pro) to 7% (V4-Flash) of V3.2's cache footprint. Fourth, CSA2 in DeepSeek-V4.1-Flash (September 10, 2026), which adds the third axis nobody had dared: reusing KV entries across layers, storing the cache in 4-bit MXFP4, and replaying instead of persisting sliding-window state — 890 bytes per token in HBM, one-eighth the persistent SSD footprint of V4-Flash.

The paper is written to be read once, linearly, and then used as a reference. Part I builds the foundation: why decode is memory-bound, how to compute cache bytes by hand for any architecture, and what DeepSeek's 2023 models cost before any of this existed. Part II covers the latent revolution of 2024, from the MLA paper's mathematics to the disk-caching business decision that turned compression into a pricing weapon. Part III covers the sparsity turn of 2025, when the question changed from "how do we store tokens more cheaply" to "which tokens do we need at all." Part IV covers the V4 era through September 2026, then compiles the evidence: master tables, a full timeline, competitor comparisons against GQA, sliding-window, and NSA-family designs, and deployment recipes for vLLM, SGLang, and LMCache. Readers who only want the numbers can jump straight to Chapter 15; readers who only want the deploy commands can jump to Chapter 16.

1. The Cache Problem: Why KV Memory Decides Everything

Attention needs the past. The past needs memory. Memory is the bottleneck.

Transformers generate tokens one at a time, but attention is defined over the entire past. When a model attends, every query must be compared against every key that came before it, and every surviving comparison must fetch its value. Recomputing all of that for every new token would be quadratic hell, so inference engines cache the keys and values once and reuse them: that store is the KV cache. It is the model's working memory of the conversation, and its size is brutally predictable — it grows linearly with every token in every sequence in the batch, forever, until something gives.

What gives first is bandwidth. During decoding, the model is memory-bound: each new token requires the GPU to read the entire cache (and the active weights) from HBM, do a little math, and write one token out. An H100 ships 3.35 TB/s of HBM bandwidth; the Tensor Economics team's benchmark of a single Llama-3.3-config attention layer at batch 64 shows decode latency tracking the "total bytes moved ÷ 3.35 TB/s" line almost exactly, with compute essentially free by comparison. This is why the chapter of this paper you are reading exists: at long context, the cache is the workload. A model whose cache is 10× smaller is, to first order, a model whose decode is 10× cheaper — and whose batching headroom, context ceiling, and offload options all improve at the same time.

Agents make it worse in a specific, structural way. The V4.1-Flash report describes the new reality plainly: long-horizon agents have made workloads input-heavy. A coding agent re-submits a growing repository transcript on every tool call; a browsing agent accumulates page dumps; a reasoning model keeps its own chain of thought in context. Input-heavy means cache-heavy: the prompt is processed once (prefill, compute-bound) but then sits in the cache being re-read on every single decode step (memory-bound) for as long as generation lasts. Multiply by continuous batching — the technique that makes serving profitable by packing dozens of concurrent requests into one GPU — and the cache's linear growth becomes the wall. Weights load once and amortize across the batch; the cache grows with the batch. At large combined sequence lengths, cache traffic exceeds weight traffic and never looks back.

DeepSeek's response to this wall is the through-line of this paper, and it is notably architectural rather than infrastructural. Other labs met the cache with bigger HBM pools, paged memory management (vLLM's PagedAttention), prefix cache reuse, or aggressive speculative decoding. DeepSeek did those too, but its signature move was to keep attacking the bytes per token inside the model itself, generation after generation:

Why should you care, if you are not DeepSeek? Because the economics propagate. When the cache shrinks, a fixed fleet of GPUs serves more concurrent users at longer contexts, and the price floor drops with it. DeepSeek's API cache-hit price fell from $0.014 per million tokens in 2024 to $0.006 per million at V4.1-Flash's launch — while the cache-hit discount versus a cache miss deepened from 10× to 50×, a direct consequence of cheaper persistent caches. The Tensor Economics analysis made the strategic case explicit: sparse attention is what makes "Claude Code-like" long-context coding products viable with positive gross margins, and DeepSeek's own V4.1 announcement states it in one line — cache-hit charges often account for a large share of agent costs, and compressing the cache cuts those costs significantly. The cache is not an implementation detail. It is the unit of account for the entire agentic era.

2. The Arithmetic of Cache Memory

One formula, three precisions, and the three axes on which every DeepSeek innovation operates.

Before the history, the mathematics — because every claim in this paper reduces to it. The KV cache size for a standard multi-head or grouped-query attention transformer is:

bytes_per_token = 2                            # K and V, both cached
                * n_kv_heads                  # key/value heads (shared in GQA/MQA)
                * head_dim                    # dimension of each head
                * n_layers                    # every transformer layer caches its own
                * bytes_per_element           # 2 for bf16, 1 for fp8, 0.5 for fp4

The factor of 2 is the K/V pair; the per-layer multiplication is the silent killer — a 61-layer model stores 61 independent copies of every token's past. Work it for three familiar configurations in bf16 and the scale of the problem appears immediately:

Two observations fall out of this table. First, MHA is not a serious option at frontier scale: 3.84 MB per token means a single 32K-context request would need 126 GB of cache before the weights arrive. Second, GQA's 8× head sharing only buys you to ~240 KB per token — still 7.9 GB for one long-context request. The gap between "GQA is fine" and "GQA is fine at 1M tokens" is where the entire DeepSeek program lives: 240 KB × 1M tokens × a commercial batch is measured in petabytes.

The precision ladder

The second lever is bytes_per_element. Halving it halves the cache outright, with arithmetic that needs no cleverness at all — only accuracy management. DeepSeek walked this ladder deliberately: V2 and V3 cached latents in bf16 by default; V3.2's production kernels moved the latent cache to fp8 (512 bytes) while keeping the rotary key in bf16 "for accuracy," a split the vLLM team documented entry-by-entry; V4 went to fp8 for most entries with fp4 reserved for the indexer's keys; and V4.1-Flash completed the descent with quantization-aware training in the OCP MXFP4 format so the 4-bit cache survives contact with real workloads. The ladder matters because it stacks multiplicatively with every architectural trick: an architectural 20× times a precision 4× is 80×, for free.

The three axes

Everything DeepSeek has ever done to this cache reduces to three multiplicative axes, and the V4.1-Flash paper finally named them explicitly when describing CSA2's design goals:

The worked example that frames everything

Fix one benchmark in memory, because this paper returns to it in every part. Take DeepSeek-V3's published attention geometry: 128 heads, head_dim 128, 61 layers, and the MLA latent of 512 + 64 = 576 cached elements per layer. In bf16, the Tensor Economics computation gives 4.0 MB/token for counterfactual MHA, ~500 KB for GQA-8 at the Llama-3 ratio, and 70 KB for MLA — a 57× and 7× reduction respectively. Now project to a production decode step at 131K context: dense MLA reads 5.2 GB per step (656 bytes × 131,072 tokens × 61 layers); DSA reads 1.1 GB; V4 reads a fraction of that; V4.1-Flash reads 890 bytes × some bounded selection × a layer count that shares entries across halves of the network. The entire history of this paper is visible in that one progression — and every one of those numbers will be re-derived, with sources, in its own chapter.

With the formula and the axes in hand, we can finally meet the baseline: the 2023 models where none of these levers had been pulled, and the cache was exactly as large as naive attention made it.

3. Before MLA: The 2023 Baseline

DeepSeek LLM 7B and 67B: a textbook MHA/GQA pair, 480 KB and 380 KB per token, and the problem that motivated everything after.

DeepSeek's first-generation language models, released in November 2023 and documented in DeepSeek LLM: Scaling Open-Source Language Models with Longtermism (arXiv:2401.02954), are deliberately unglamorous. The two models — a 7B and a 67B dense transformer — were built to prove the lab could scale the standard recipe with unusually careful data hygiene (67B surpassing LLaMA-2 70B on code, math, and reasoning benchmarks with a smaller training corpus), not to innovate on attention. And that makes them the perfect baseline: they show exactly what the cache costs when nobody has touched it.

The attention designs were the era's defaults, as the team's own release notes state plainly: the 7B model uses Multi-Head attention (MHA) while the 67B model uses Grouped-Query Attention (GQA). Plug the published configurations into the Chapter 2 formula and the baseline appears:

One detail deserves a raised eyebrow: the 7B model caches more per token than the 67B. A quarter of the layers of the big model, but four times the KV heads — MHA's 32 heads at 16 KB per layer beat GQA's 8 heads across 95 layers. The cross-over is the whole GQA pitch in miniature: at 7B scale nobody feels it (480 KB × the 4K context of that era is 1.9 GB, awkward but survivable on a single GPU), while the same context on the 67B is 1.5 GB of pure cache on top of 134 GB of bf16 weights. The era's context lengths — 4K native, extendable to 32K with YaRN-style interpolation — were chosen precisely because the cache arithmetic said so. Nobody ships a 128K window when the per-token bill is 380 KB.

It is also worth recording what the lab was learning in parallel, because it explains the architecture that followed. The DeepSeek LLM paper itself ran GQA-Int8 ablations — 8 KV heads in int8 storage, cutting the 67B's cache to ~194 KB/token, and finding it "surprisingly strong" — an early signal that the team was already thinking about cache bytes as a first-class design variable rather than an afterthought. And in January and February 2024, two companion papers arrived that built the other half of the future: DeepSeekMoE (arXiv:2401.06066), which split experts fine-grained and added a shared expert to make sparse FFNs cheap and stable, and DeepSeekMath (arXiv:2402.03300), which introduced GRPO, the reinforcement-learning algorithm that would later make R1 famous. The V2 recipe — MoE for cheap computation, MLA for a cheap cache, RL for the finishing — was assembled from these parts.

The baseline also established the lab's economic reflex, which never changed afterward: the DeepSeek LLM paper's title word longtermism referred to training-data quality, but the engineering culture it named — spend carefully, keep the recipe reproducible, publish everything — is the same one that later priced V2's API at a fraction of competitors and priced V4.1-Flash's cache hits at half a cent per million. In attention as in training, DeepSeek's instinct was always to make the expensive thing small first.

4. DeepSeek-V2: MLA Changes Everything

May 2024: 236B parameters, 21B active, and a KV cache compressed into a single latent vector per token.

DeepSeek-V2 (arXiv:2405.04434, released May 2024) is one of the most consequential architecture papers of the decade, and its abstract compresses the entire business case into three numbers: saves 42.5% of training costs, reduces the KV cache by 93.3%, and boosts the maximum generation throughput to 5.76× — all comparisons against the lab's own DeepSeek 67B. The model itself is a 236B-parameter MoE with 21B activated per token, 60 transformer layers, native 128K context (via the MLLA architecture plus YaRN-style extension from a shorter pretraining window), and 8.1T pretraining tokens. But the number everyone remembers is the cache.

The idea in one paragraph

MQA shares one K/V head across all query heads (tiny cache, degraded quality). GQA splits the difference (smaller cache, small quality cost). MLA refuses the trade: instead of sharing heads, it caches a compressed latent representation and reconstructs per-head keys and values from it on demand. Per token, per layer, the cache holds exactly two things: a 512-dimensional compressed latent c_KV shared by all heads, and a single 64-dimensional RoPE-encoded key k_rope (the decoupled positional key, Chapter 5). That is 576 cached elements versus 32,768 for the counterfactual MHA — 1.76% of the bytes, with the V2 paper's own ablation table showing MLA outperforming MHA on hard benchmarks, not merely matching it.

The arithmetic, honestly done

Run the Chapter 2 formula across the full 60-layer stack and the totals fall out cleanly: MHA 3.84 MB/token, GQA-8 240 KB, MLA 67.5 KB (the paper rounds to 70 KB). MLA's per-token footprint is thus 56.9× smaller than MHA at identical geometry — which matches the ~57× theoretical HBM-traffic reduction the paper derives for decoding — and 5.6× smaller than the 67B's GQA-8 across its 95 layers (389 KB → 67.5 KB, an 82.2% reduction). The abstract's headline 93.3%, meanwhile, is DeepSeek's own accounting against the 67B under its stated comparison conditions; independent arithmetic at face-value configs lands between 82% and 98% depending on the baseline chosen (the V2 report's MHA-vs-MLA framing implies the upper end, the GQA framing the lower).

Why the latent survives reconstruction

The obvious objection to MLA is information loss: a 512-dimensional vector cannot hold what 32,768 dimensions held. The answer is that the reconstruction is not lossy compression of K and V — it is a learned low-rank factorization. The model is trained end-to-end to place everything attention needs into the latent, so the "compression" is baked in during pretraining rather than applied after the fact. The V2 paper's ablations make the empirical case at two scales (16B on 1.33T tokens; ~250B on 420B tokens): MLA beats MHA on hard benchmarks while requiring a fraction of the cache, and — the detail that sealed the design — it also beats GQA and MQA on quality while beating them on size. The report's Figure 12 comparison, reproduced across dozens of explainers since, shows the four mechanisms on a single axis where MLA sits alone in the top-left corner: maximum quality, minimum cache.

What it meant at the time

V2's 128K context arrived at 67.5 KB/token instead of 240 KB: at full window, one request's cache falls from ~31.5 GB (counterfactual GQA-8) to ~8.9 GB, and — because decode traffic scales with cache bytes — the measured maximum generation throughput rose 5.76× against the 67B. The launch-era API pricing made the savings visible to customers immediately, and the lab followed with a systems-level complement that turned prefix reuse into a near-free operation (Chapter 6). But the deeper consequence was strategic: V2 proved that a frontier-quality model could carry an order-of-magnitude cache advantage, which is precisely the property that later made a 671B V3 affordable to train (Chapter 7), a reasoning RL run viable on top of it (Chapter 8), and a 1M-token V4 conceivable (Chapter 12).

There is one more thing V2 did that gets less credit than it deserves: it made the cache compressible by construction for everything that followed. A 576-element latent is a fixed, compact, well-conditioned target for quantization; a pile of per-head K/V tensors is not. When V3.2 moved the latent to fp8 (Chapter 10) and V4.1 trained it in 4-bit (Chapter 14), they were spending savings V2's design had already banked. The next chapter opens that box: the actual mathematics — the projections, the absorption trick, and the decoupled RoPE that makes it all invertible.

5. Inside MLA: The Mathematics

Down-projections, up-projections, the absorption trick, decoupled RoPE, and the two execution modes that make prefill and decode each optimal.

MLA is six matrices and one idea. This chapter is the complete derivation, using DeepSeek-V3's published configuration as the running example (identical geometry to V2's attention except for layer count), because those are the values every serving kernel in the ecosystem is compiled around today. Follow the symbols once here and Chapters 10, 12, and 14 read like plain prose; the dimensions are also collected in Table 5.1 for reference.

5.1 — Compress, then cache

For each token's hidden state h (7168 dims in V3), MLA computes a compressed latent for K and V jointly:

c_KV = W_DKV @ h          # W_DKV: 7168 x 512   -> the ONLY thing cached for content
# c_KV: 512 dims per token per layer, shared by all 128 heads

# reconstruction (used in prefill / training):
K_C = W_UK @ c_KV         # 512 -> 128 heads x 128 dims
V_C = W_UV @ c_KV         # 512 -> 128 heads x 128 dims

The latent is the cache: 512 floats instead of 32,768. Queries get the same treatment with a wider bottleneck — c_Q = W_DQ @ h with W_DQ projecting 7168 → 1536 — but queries are never cached, so their compression is purely a FLOPs/regularization play. The asymmetry (512 for cached KV, 1536 for transient Q) is deliberate: you spend dimensions where persistence costs money.

5.2 — The RoPE problem and the decoupling

Rotary position embeddings rotate queries and keys by position, and rotation does not commute through matrix multiplication. If RoPE sat between c_KV and the reconstructed key, you could never fold W_UK into anything — the rotation would have to be applied to the full reconstructed key at every step, re-materializing exactly the tensor MLA exists to avoid. DeepSeek's solution, decoupled RoPE, splits every head's key into two parts that never mix:

• The NoPE part (content, 128 dims per head): derived from the latent, carries semantic content, never rotated, and fully absorbable.
• The RoPE part (position, 64 dims): a single shared key k_rope = RoPE(W_KR @ h) computed directly from the hidden state (7168 → 64), cached alongside the latent, and broadcast to all 128 heads.

The asymmetry is the elegant part: each head keeps its own 64-dim RoPE query (heads specialize positionally at zero cache cost) while sharing one RoPE key across all heads — saving 128× on the positional cache. Per head, the attention score is simply the sum of the two dot products: q_nope·k_nope + q_rope·k_rope. Total cache per token per layer: 512 (NoPE latent) + 64 (shared RoPE key) = 576 elements — the number that recurs through every model from V2 to V3.2.

5.3 — The absorption trick

Decode's cost is bytes read, so the game is: attend against the latent directly, never reconstruct K. Matrix associativity makes it legal:

q_nope @ k_nope^T
  = (c_Q @ W_UQ_nope) @ (c_KV @ W_UK_nope)^T
  = c_Q @ [ W_UQ_nope @ W_UK_nope^T ] @ c_KV^T
             \______ precomputed once: W_absorb ____/      # 1536 x 512

# decode attention = SDPA(q_absorbed, c_KV) + SDPA(q_rope, k_rope)
#                  ~ MQA-shaped: one shared 576-elem "key" per token

Precompute W_absorb = W_UQ_nope @ W_UK_nope^T once at engine warmup, and every decode step's key path is a plain matrix multiply against the 576-element cache — structurally identical to multi-query attention, which is why FlashMLA and even Apple's MLX run MLA decode as ordinary scaled_dot_product_attention after absorption. The value path has a mirror trick available (folding W_UV @ W_O), but production engines decline it: the fused matrix would be 3.7× larger than keeping the two separate (a weight-traffic regression that would eat the cache savings), so W_UV and W_O are applied as two small post-attention operations. Engineering is knowing which algebra to leave on the table.

5.4 — Two modes, one mechanism

Because prefill is compute-bound and decode is memory-bound, MLA runs in two mathematically equivalent regimes — the V2 paper's own framing, confirmed by the V3.2 infrastructure notes:

5.5 — The configuration, for the record

Two properties of this design earned it the industry's respect. First, quality parity: the V2 ablations show MLA outscoring MHA (and GQA, and MQA) at matched training budgets — low-rank projection turned out to be a mild regularizer of attention rather than a bottleneck, which is why no DeepSeek model since has felt the need to widen the latent. Second, portability: because absorbed decode is plain SDPA, MLA runs anywhere a matmul runs — MLX on Apple Silicon, consumer runtimes, research code — with the full compression and no custom kernel. The custom kernels (FlashMLA, Chapter 8) exist to go faster than SDPA, not to make the mechanism possible.

6. Cache on Disk: The Systems Layer

August 2, 2024: prefix reuse becomes an automatic, near-free feature — and the cache turns into a pricing instrument.

Architecture compressed the bytes; the next DeepSeek move compressed the tokens you pay for twice. On August 2, 2024, the API announced Context Caching on Disk: every request's prompt KV is written to a disk-backed store, and any future request whose prompt shares a prefix with a cached one reuses that prefix's KV instead of recomputing it. Billing is automatic — no code changes, no opt-in — and cache hits are billed at a 10× discount: $0.014 per million tokens against the $0.14 input price of the era, a cut of up to 90% on the cached portion of a bill. The service shipped with unlimited concurrency and, in the announcement's own framing, was aimed directly at the repeating-prefix patterns of chat and agent workloads: system prompts, tool schemas, few-shot exemplars, conversation history.

The arithmetic for an agent is stark and worth internalizing, because it is the template for every pricing decision DeepSeek has made since. A coding agent with a 32K-token system prompt, tool definitions, and repo context, making 500 calls in a session, re-sends that 32K prefix 500 times: 16M input tokens of pure repetition. Without caching that is 16M full-price tokens plus the prefill compute; with an effective disk cache it is one miss plus 499 hits at a tenth (later a fiftieth) of the price. This is why the announcement could honestly claim "up to 90%" — and why cache-hit ratios became a first-class number to monitor in the API's usage dashboard (the docs expose hit/miss fields on every response). The agent era's economics were being quietly rewritten nine months before anyone said "agentic workloads" in a model announcement.

The model lineage around this systems work kept improving while the attention stayed frozen: DeepSeek-V2.5 (September 5, 2024) merged the V2 chat and DeepSeek-Coder lines into one model with the same MLA cache and 128K context, and the V2.5-1210 point release (December 10, 2024) tuned alignment and function-calling — the property agents actually felt. Through all of it, the per-token cache never changed: 576 elements, ~70 KB. The compression and the reuse were now both in place. What remained was scale — and the 671B model that would make MLA carry more parameters than any open model in history.

7. V2.5 to V3: Scaling the Latent

December 26, 2024: 671B parameters, 37B active — and the exact same 576-element cache, now carrying the biggest open model on Earth.

DeepSeek-V3 (arXiv:2412.19437) is the model that made the lab globally famous, and its treatment of the KV cache is a study in confidence: it changed nothing. The technical report adopts MLA and DeepSeekMoE "thoroughly validated in DeepSeek-V2" as-is — the same 512+64 latent, the same 128 heads, the same decoupled RoPE — and simply scales around them: 671B total parameters, 37B active per token, 61 layers (one more than V2), 14.8T pretraining tokens, and an auxiliary-loss-free load-balancing scheme that keeps the MoE routing healthy without gradient interference. Training consumed 2.788M H800 GPU-hours (about $5.6M at rental rates, the report notes) with zero irrecoverable loss spikes.

That stability is the point of this chapter. V3 was the test of whether MLA's low-rank cache survives a 10× parameter scale-up, 128K native context, FP8 mixed-precision training, and multi-token prediction heads — and it passed without modification. The Tensor Economics recomputation of the cache arithmetic at V3's exact geometry (128 heads, 128 head_dim, 61 layers) makes the inherited advantage precise:

Concretely: a fully-loaded V3 request at 128K context carries ~9.2 GB of bf16 latent cache (70.3 KB × 131,072 — or ~4.6 GB at fp8), where the GQA-8 equivalent would need 65 GB and MHA an impossible 524 GB. The same property composes with batching: at batch 64 with 4K tokens each, MLA's combined cache (~18 GB bf16) fits beside the 671B FP8 weights on an 8×H200 node with room to grow, while the GQA equivalent would already be thrashing. Every serving economics number published for V3 — the record-setting 149K tokens/second decode benchmark, the sub-dollar input pricing, the 3× R1 price cut that followed — has this table underneath it.

V3 also planted the seed of the next chapter quietly: with MoE experts sharded across GPUs (two routed experts plus one shared expert per accelerator in the production EP layout), the FFN weight traffic per decode step is smaller than a dense model's — which means attention cache traffic dominates the memory budget earlier, not later. The Tensor Economics analysis states it bluntly: the real-world case for attacking the cache's read path is stronger than the dense-MLP figures suggest. MLA had solved entry size; decode still read every cached token, every step, forever. That wall now had a name and a date.

8. R1, FlashMLA & the Ecosystem

January–February 2025: the reasoning model that stress-tested MLA, and the open kernels that turned one lab's trick into an industry standard.

DeepSeek-R1 (January 20, 2025) was the model that put the lab on every front page, and from the KV cache's perspective it was the harshest test the latent would ever face. R1 is built on V3's 671B/37B architecture — same MLA, same 61 layers, same 576-element latent — and then trained with GRPO reinforcement learning to think in long, self-generated chains. Reasoning traces are the input-heavy workload par excellence: tens of thousands of tokens of self-dialogue that stay pinned in the cache while the model generates tens of thousands more. If the low-rank latent were a quality bottleneck, RL would find it — RL is a ruthless optimizer of whatever the model can and cannot express. R1's results (matching OpenAI o1 on math, code, and reasoning benchmarks) showed the compressed cache carried a frontier reasoning model without visible strain, and the May 28, 2025 R1-0528 update, with its deeper thinking traces and better tool use, re-confirmed it under heavier loads.

The second ecosystem event of the winter was the one that mattered structurally: on February 24, 2025, DeepSeek open-sourced FlashMLA — its production MLA decode kernels for NVIDIA Hopper GPUs, developed alongside the team's DeepGEMM FP8 GEMM library. The release targeted multi-batch, split-KV serving with paged cache organization (block size 64, matching the page geometry V3.2 would later require), BF16 decode optimized for the H3D regime (300–600 GB/s effective bandwidth conditions), and the now-canonical 576-element latent layout. Together with the earlier NSA paper this constituted a deliberate infrastructure reveal: the attention mechanism and the optimized kernels to run it were now public property.

The adoption wave that followed is the cleanest evidence of MLA's engineering quality. Kimi K2 (Moonshot AI's trillion-parameter MoE) adopted MLA wholesale. SGLang, vLLM, and TensorRT-LLM all landed FlashMLA-backed MLA decode within weeks; the MLX ecosystem ran absorbed-MLA decode as plain SDPA on Apple Silicon with the full 57× compression and zero custom code; LMCache built its KV-offload storage layer around the latent's compact layout. Later in 2025 and 2026, Z.ai's GLM-5/5.1 went further and adopted DSA itself — the first major external validation that DeepSeek's sparse path was the industry's path. The whale had turned its internal cache program into an industry standard, one open kernel at a time.

By the middle of 2025, every piece was on the table except the one that mattered: using it all in a shipping model. The interim release — V3.1 — would be the last DeepSeek model to read its entire cache on every decode step.

9. V3.1-Terminus: The Last Dense-Attention Model

August 21, 2025: hybrid thinking modes, a 3.3× longer context-extension phase — and the final iteration of full-context MLA.

DeepSeek-V3.1 (August 21, 2025), dubbed Terminus, is a consolidation release, and it deserves a precise place in this history because it marks the end of an era. Architecturally it is V3's skeleton — 671B/37B, MLA, 61 layers, 70 KB latent per token — refined in three directions: a hybrid thinking mode (one model switching between a fast non-thinking path and a deliberate thinking path, an Anthropic-style API format), a 128K context window whose final extension phase was stretched 3.3× longer than V3's to harden long-context behavior, and an agent-oriented post-training with the tool-calling reliability that V2.5-1210 had only gestured at. A September 22 update rounded out function-calling and interleaved thinking.

What Terminus did not do is touch the cache. Every decode step still read all 656 bytes per token per layer (in the fp8-with-bf16-RoPE serving layout the ecosystem had converged on), for every cached token, in every layer — the arithmetic of the Chapter 7 warning, now stretched across 128K windows that agents actually used. The V3.1 release notes contain no cache claims, and that silence is the story: by August 2025 the entry-size axis was exhausted. The latent was already near the information floor for full-attention quality; squeezing bytes further meant changing which tokens the model reads.

10. V3.2-Exp: The Lightning Indexer

September 29, 2025: DeepSeek Sparse Attention — the first fine-grained sparse attention in a frontier model, a 656-byte latent that is only read for 2,048 tokens, and an API price cut of more than half.

DeepSeek-V3.2-Exp (arXiv:2509.17722) did with a single architectural change what two years of kernel optimization could not: it broke the linear relationship between context length and decode cost. The mechanism, DeepSeek Sparse Attention, keeps MLA's latent cache exactly as Chapter 5 left it — same 576 elements, same 61 layers — and inserts one new module in front: a lightning indexer that scores every cached token cheaply, followed by a top-k selection that decides which tokens the expensive attention will actually touch. The base model was V3.1-Terminus, retrained to work with the new mechanism; the official launch announcement arrived with the API's prices cut by more than 50%.

10.1 — The indexer, precisely

The indexer is deliberately dumb and deliberately small. It runs in multi-query mode — 64 heads, all sharing one cached 128-dimensional key per token — and computes nothing but dot products:

I(t, s) = sigmoid( tau * sum_j w_j * ReLU( q_tj . k_s ) )
#          \_______/    \__ head weights __/ \_ shared key _/
# 64 query heads, 128-dim shared index key, one score per (query, token)
# - ReLU: only positive evidence counts (no softmax over all tokens)
# - w_j: learned per-head weights, combined into ONE relevance score
# - sigmoid(tau * .): bounded gate; tau is a learned temperature
#
# top_k = 2048:  the k tokens with highest I(t, s) get full MLA attention

Because the indexer stores keys only — no values, no full attention — and shares its key across heads MQA-style, its per-token footprint is minimal: 128 bytes for the fp8 key plus a 4-byte fp32 block scale, 132 bytes per token per layer, one-fifth of the latent's 656. The vLLM Day-0 support post documents the production layout exactly, including the per-block fp8 scales and the block size of 64 that FlashMLA's paged layout had already standardized. The indexer is trained jointly with the model — the V3.2 report describes learning it during a continued-pretraining phase — so the selection policy is the model's own learned judgment, not a heuristic bolted on afterward.

10.2 — The two caches, byte by byte

The asymmetry between the two caches is the entire invention. The indexer must see every token (it cannot know what is irrelevant without scoring it), but it sees them at 132 bytes instead of 656. The main attention reads the full-fat latent only for the 2,048 tokens the indexer selected — a fixed cost that does not grow whether the context holds 10,000 tokens or 131,000. Storage still grows with N (nothing is evicted; the selected set changes every step), but read traffic, the thing that determines decode time on a bandwidth-bound GPU, is now:

dense_Mla(N) = 656 B * N      * 61          # everything, every step
dsa(N)      = 132 B * N      * 61          # indexer sees all, cheap
             + 656 B * 2048  * 61          # latent read: FIXED (82 MB total)

# N = 131,072 (128K context):
#   dense: 656 * 131072 * 61          = 5.24 GB per step
#   DSA:   132 * 131072 * 61 + 82 MB  = 1.06 + 0.08 = ~1.1 GB per step
#   ratio: ~4.8x less traffic -> near-constant decode time at depth

That ~5× traffic reduction at 131K — independently verified by the Tensor Economics H100 benchmarks, where DSA's decode latency curve flattens while dense MLA's climbs linearly — is what the report's headline efficiency claims are made of. DeepSeek reported substantial long-context efficiency improvements in both prefill and decode (the paper's prefill arithmetic-intensity reduction is roughly 20×, and measured decode speedups at 128K are around 3× once the indexer's own cost and the sparse-kernel overheads net out), and there is a systems dividend that benchmarks undersell: because per-layer attention time becomes nearly constant, the dual micro-batch overlap that hides MoE all-to-all communication becomes predictable enough to schedule tightly — the property the production DeepSeek infra docs highlight, and the one that makes wide expert-parallel serving stable at commercial batch sizes.

10.3 — What it cost and what it bought

The quality story is the part the industry watched most closely, and it is clean: across the V3.2-Exp evaluation suite, benchmark scores match V3.1-Terminus — the same model with dense attention — to within noise on standard reasoning and knowledge tasks, with long-context retrieval tasks (where full attention's exhaustive scan was thought to matter most) holding at or above the dense baseline. Sparsity, trained properly, was free. What it bought was commercial: the launch announcement's price table — input cache-miss $0.28/M (from $0.56), cache-hit $0.028/M, output $0.42/M (from $1.68) — a cut of 50% on input and 75% on output, effective the same day. The Tensor Economics commentary called the mechanism out as the enabler of profitable long-context coding products; the pricing table is that thesis with numbers attached.

10.4 — Day-0 kernels and what the support post reveals

The engineering lift to serve DSA was nontrivial, and the vLLM team's Day-0 post is the best public record of why. Three cache layouts now coexist (latent, indexer keys at per-block granularity, page table mappings between them); continuous batching must track separate prefill/decode paths for the indexer; and the top-k over a materialized (q, n, h) logits tensor wants fusion at scale — vLLM shipped a fused kernel taking design cues from DeepSeek's own TileLang reference. The compute path uses new DeepGEMM routines (fp8_mqa_logits for indexer scoring, causality bounded by per-query start/end context markers) and a new sparse attention kernel in FlashMLA, with out-of-the-box Blackwell support landed in collaboration with NVIDIA (B200 and GB200 from day zero). SGLang's simultaneous Day-0 post told the same story from its side. The message to the industry was unmistakable: when DeepSeek ships an attention mechanism, it ships the kernels too, and the engines follow within hours.

# 8x H200 / 8x B200 / 16x H100; block size 64 enforced by the layout
vllm serve deepseek-ai/DeepSeek-V3.2-Exp \
    --tensor-parallel-size 8 \
    --trust-remote-code
# indexer K cache + MLA latent cache are allocated automatically;
# fp8 KV is the model's native serving precision (656 B + 132 B layout)

V3.2-Exp was released as an experiment and flagged as such — weights on Hugging Face, kernel stack open, evaluations honest about the frontier. Ten weeks later the experiment graduated.

11. V3.2 & Speciale: Sparsity Grows Up

December 1, 2025: the production release — gold medals at four olympiads, reasoning that survives tool calls, and the price floor that defines the V4 baseline.

DeepSeek-V3.2 (arXiv, December 2025) took the experimental sparsity out of the lab and into the flagship line, and the official release framed it around capability rather than cost: "world-leading reasoning, thinking in tool-use, and gold-medal performance in IMO, CMO, ICPC & IOI 2025." The V3.2-Speciale sibling — a specialist trained for competition mathematics and informatics — scored 35/42 on the International Mathematical Olympiad 2025, gold-medal level, and matched the same standard at IOI, CMO, and the ICPC World Finals. A model that attends to only 2,048 tokens per query had reached the ceiling of formal problem-solving. For anyone still arguing that sparsity trades quality for speed, December 2025 closed the case.

The release's other pillar, thinking in tool-use, is the cache story wearing agent clothes: reasoning traces are now preserved across tool-result rounds — the model keeps its chain of thought while executing calls instead of flushing context at every boundary — which multiplies context occupancy per session and makes DSA's flat decode curve a revenue-relevant property rather than a benchmark curiosity. The same release introduced the massive agent-training data synthesis pipeline the announcement credits for its tool- reasoning gains, and the community price tracking recorded output at $0.42 per million tokens with cache hits at a tenth of input — the levels V3.2-Exp had established, now with a stable, non-experimental, MIT-licensed model family behind them (independent trackers pegged the era's total cost-of-ownership reduction around 70% against V3.1).

One year of sparsity had rewritten the cost structure of long context. But DSA still stored everything and still ran an O(N) indexer scan per query; prefill still paid full quadratic compute on input-heavy agent transcripts. The V4 family, previewed four months later, attacks both residuals at once — and finally moves from reading fewer tokens to storing fewer tokens.

12. V4: CSA + HCA, the 1M Era

April 24, 2026: two checkpoints, one million tokens of context, and a hybrid attention stack that pools the past before it caches it.

DeepSeek-V4 Preview (arXiv:2606.19348, open-sourced April 24, 2026) is the release where the KV cache stops being a compressed record of the past and becomes a summary of it. Two MoE checkpoints shipped together: V4-Pro, 1.6T total parameters with 49B active, and V4-Flash, 284B total with 13B active — both carrying a 1M-token context window. The headline comparisons were framed against V3.2 at one million tokens: V4-Pro uses 27% of the single-token inference FLOPs and 10% of the KV cache memory; V4-Flash drops to 10% of the FLOPs and 7% of the cache. Against a conventional GQA-8 design storing bf16, the V4 cache is roughly 2% of the size. The mechanism is a hybrid of two new attention types, interleaved layer by layer.

12.1 — Compressed Sparse Attention (CSA)

CSA attacks the sequence axis directly: every 4 tokens are pooled into a single cached KV entry using softmax-gated pooling with a learned positional bias — the compressor decides, per group, how to represent four tokens in one pair of latent vectors. A lightning indexer (the V3.2 inheritance, now running its dot-product scoring over the 4×-shorter compressed stream in FP4) then picks the top-k compressed blocks per query, and attention reads only those. The causality subtlety that pooling introduces — a compressed entry aggregates future tokens into a past token's representation — is handled by a strict guard documented in DeepSeek's LMCache collaboration: entries are only compressed once they are at least 128 tokens behind the current position, so no query ever attends to information from its own future. A sliding-window branch covers the uncompressed recent tokens, whose local detail pooling would blur.

12.2 — Heavily Compressed Attention (HCA)

HCA is CSA's id-driven sibling: 128 tokens pooled into one entry, and then — crucially — no sparse selection at all. The compressed stream is short enough that every query attends densely to every compressed block. The design trade, as the LMCache engineering blog's close reading of the paper puts it, is fine-grained selectivity exchanged for total recall: DSA's top-1024 selection (V4's k, tightened from V3.2's 2,048) sees only what the indexer ranks; HCA lets everything participate at 1/128th resolution. DeepSeek's own framing is that CSA preserves selectivity while HCA provides an aggressively compressed global view — the two interleaved give the model both the ability to zoom in on specific history and a perfect (if blurry) memory of all of it.

12.3 — The layer map and the precision stack

In V4-Pro's 61-layer stack, layers 0–1 run HCA (the model's early, global "gist" passes), layers 2–60 alternate CSA and HCA, and the multi-token-prediction block at the end runs sliding-window attention only. Precision is assigned per role: FP8 for most KV entries, BF16 only for the RoPE dimensions, FP4 for the indexer's keys — the storage choices that, compounded with the pooling ratios, produce the ~2%-of-GQA figure. The indexer path itself was quantized from FP32 to BF16 during development for a 2× speedup with 99.7% top-k recall preserved, a number the report cites as evidence that selection quality is robust to precision. Training staged the mechanisms: sequence length ramped 4K → 16K with dense attention, and the sparse-compressed path switched on at a 64K stage — 32T pretraining tokens for Flash, 33T for Pro.

12.4 — The agent features that load the cache

Three post-training decisions in the V4 report target agent workloads specifically, and each one increases context occupancy — which is only rational because the cache is now cheap:

• Interleaved thinking across tool calls: where V3.2 preserved reasoning across tool-result rounds but flushed it at each new user message, V4 keeps the full reasoning history across user turns whenever the conversation contains tool calls — a coding agent remembers why it changed a file, across the whole session. Conversations without tools keep the old flush behavior to stay concise.
• Quick Instruction tokens: auxiliary decisions ("should I search?", intent classification) run as special tokens inside the main model, reusing the already-computed KV cache instead of a separate helper model re-prefilling the context — orchestration overhead amortized into cached prefix bytes.
• DSec: a Rust sandbox platform exposing function calls, containers, and more behind one SDK, built for RL rollouts against real tool environments — the training-time engine that produced the tool-use robustness the benchmarks measure.

The open-source community's first-week measurements filled in the economics. LMCache's analysis concluded V4's cache is roughly 10× smaller than V3.2's end-to-end, translating to 2–3× higher token-generation throughput on the same hardware and a similar factor on price — and its deployment recipe (Chapter 16) landed day-one with the fp8_ds_mla KV dtype and a deepseek_v4 tokenizer mode that vLLM requires for the hybrid layout. The one rough edge: the vLLM development branch of the era mis-dispatched the FP4 MoE experts, and the recipe pinned to tagged releases — a reminder that a hybrid attention stack ships as one system: cache geometry, indexer kernels, expert routing, and all.

# sparse-MLA backends + fp8_ds_mla KV kernels: use the tagged vLLM release
vllm serve deepseek-ai/DeepSeek-V4-Flash \
    --tensor-parallel-size 8 \
    --enable-expert-parallel \
    --kv-cache-dtype fp8_ds_mla \
    --trust-remote-code \
    --tokenizer-mode deepseek_v4

# optional native MTP speculative decoding (validated with LMCache):
#   --speculative-config '{"method":"mtp","num_speculative_tokens":1}'
# gsm8k store-vs-retrieve: 0.95 vs 0.96 (sampling stderr)
# MTP acceptance rate unchanged: 0.947 vs 0.952 with cached prefix KV

What V4 did not yet solve is visible in its own numbers: V4-Flash still needed 7% of V3.2's cache at 1M tokens, the per-layer copies were still per-layer, and persistent (SSD-resident) prefix caches still had to store the sliding-window state. Four months later, a 552B-parameter model with a new architecture addressed all three residuals at once — and cut the headline number to 890 bytes.

13. V4-Pro GA & the Flash Lineage

August 2026: the flagship goes general-availability, vision arrives, the ecosystem digests — and then the smallest model in the family retires the biggest one.

For four months after the April preview, the V4 line industrialized. On August 13, 2026, DeepSeek-V4-Pro-0813 reached general availability across app, web, and API — the release DeepSeek framed around agent capability, with the vendor-reported benchmark gains of the 0813 tuning round awaiting independent confirmation (as Tech Times' launch coverage duly noted) and new pricing taking effect August 16. A week later, on August 21, DeepSeek-V4-Flash-Vision-Exp brought native multimodal input to the API on the Flash chassis — the first sign that the V4-Flash cache design was cheap enough to carry image tokens at scale, since vision contexts are the most input-heavy workloads of all.

The serving ecosystem spent the same season maturing around the hybrid cache. LMCache validated its full secondary-storage integration with V4-Flash — the sparse-MLA hybrid layout (compressed MLA latents in fp8/uint8 alongside float32 indexer groups at differing block geometries) required custom handling that its connectors now ship — and published the MTP validation evidence quoted in Chapter 12: speculative decoding with a cached prefix preserves score-level equivalence (gsm8k 0.95 computed vs. 0.96 retrieved, within sampling error) and leaves the draft layer's acceptance rate untouched (0.947 vs. 0.952), with cold-vs-warm TTFT improving 6.6× when MTP runs throughout. TRT-LLM support landed alongside; SGLang remained unvalidated for the connector. The LocalLLaMA community, meanwhile, worked the other end of the curve: 256 GB workstation owners sizing V4-Flash for local serving and finding the shrunken cache decisive for what fits.

Then, on September 10, 2026, the succession happened in public. DeepSeek-V4.1-Flash launched as the smallest model of the new architecture family — and the announcement's comparison table put it ahead of V4-Pro on performance, cost, speed, and total runtime in tests by multiple parties. The Flash retired its predecessors in a stroke: deepseek-v4-flash and deepseek-v4-flash-vision-exp route to V4.1-Flash, and a phase-out notice scheduled all deepseek-v4-pro traffic to reroute at V4.1-Flash rates from September 14 — a notice the pricing page later softened ("in response to user demand," V4-Pro API service continues, billed as before). Whatever the final disposition, the direction was unmistakable: the 890-byte cache had made a 552B model cheaper to serve than a 1.6T one, and better. That model is the subject of the next chapter.

14. V4.1-Flash: The 890-Byte Cache

September 10, 2026: a causal encoder-decoder with cross-layer cache sharing, 4-bit MXFP4 KV, and bounded replay — the smallest per-token cache ever shipped in a frontier model.

The technical report's title is a mission statement: DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression. The model is a 552B-parameter multimodal MoE pre-trained on 45T tokens, supporting one-million-token contexts, and it is built around three structural commitments: a Causal Encoder-Decoder (CED) split, Compressed Sparse Attention 2 (CSA2) with cross-layer KV reuse, and 4-bit MXFP4 KV caching with quantization-aware training. The results are the paper's own headline pair: a global KV footprint of 890 bytes per token in HBM — roughly one quarter of V4-Flash's — and, via a deployment technique called SWA Bounded Replay, a persistent footprint on SSD or host memory of roughly one eighth of V4-Flash's.

14.1 — The asymmetric architecture

V4.1-Flash's 40 causal transformer layers split evenly: the first 20 act as an encoder, the last 20 as a decoder — and the split is parametric, not just positional. The encoder activates 8B parameters per token; the decoder activates 16B. Decoder layers do not recompute their global KV from their own hidden states at all: their global KV entries are projected directly from the encoder's final hidden state H_L/2. The prefill consequence is arithmetic: complexity falls from O(N·L) to approximately O(N·L/2 + n_win·L/2), where n_win is the sliding-window size — effectively halving the compute for input-heavy workloads, the exact profile of long-horizon agents that the report identifies as the motivating workload. A model that reads a million tokens per task has just been re-proportioned to spend its FLOPs where its revenue is.

14.2 — CSA2: the three axes, all at once

CSA2 is the mechanism the V4.1 report describes as minimizing cache storage and computation "across three dimensions" — the taxonomy from Chapter 2, finally complete:

Cross-layer reuse is implemented as three operating modes that different layers run in. Full mode: the layer computes its own main K and V, projects an indexer key, and generates fresh top-k indices — the expensive, authoritative pass. Reindex mode: the layer reuses the main KV and the indexer keys from a previous layer but computes its own indexer query, selecting a different top-k subset — new selection, zero re-storage. Reuse mode: both the main KV and the top-k indices are inherited outright from a preceding layer, eliminating essentially all index-related computation. Stacked across 20 decoder layers, the modes mean one authoritative KV computation serves several layers of attention, with only queries (which are never cached) varying per layer.

The residual risk — that shared indices starve some layers of the tokens they need — is handled by the Hierarchical Sparse Indexer: the first Full-mode decoder layer selects a blockwise candidate pool, and subsequent Reindex-mode layers score only candidates within that pool. The report's claim is the one that makes 1M context boring: per-query indexing cost stays bounded whether the context is 10,000 tokens or 1,000,000. Selection no longer scales with history.

14.3 — The 890 bytes, accounted

The arithmetic of the headline now closes. Per token: MXFP4 main KV entries (4 bits per element across the pooled, head-shared latents), a 4×-pooled sequence representation inherited from the V4 compressor family, an indexer's worth of small keys — and the layer dimension divided by the reuse factor of the Full/Reindex/Reuse schedule. The result, always resident in HBM: 890 bytes per token, against ~390,000 bytes for DeepSeek-V1 at 67B scale — a 438× reduction in the report's own retrospective framing. Composed with the CED prefill halving and the bounded indexer, the report plots decode computation remaining nearly constant as context scales to one million tokens.

14.4 — SWA Bounded Replay: deleting the local cache

The slickest systems trick in the release targets persistent storage. Hybrid attention needs a sliding-window cache for recent tokens, and that cache traditionally must be persisted (to SSD or host DRAM) so a resumed request can rebuild its state. V4.1-Flash declines: when a request resumes and the sliding-window KV is missing, the system replays only the most recent n_win tokens to approximately reconstruct the local state — accepting a bounded approximation on local detail rather than paying to persist exact copies. The global cache (long-term dependencies) remains worth persisting exactly; the local one does not. Persistent footprint: one eighth of V4-Flash's, and the agent-billing consequence is the announcement's own line — cache-hit charges are a large share of agent costs, and compressing the cache cuts them significantly.

14.5 — Engram and the rest of the machine

Two further mechanisms round out the architecture. Engram is a conditional memory module of 196B parameters using multi-head hashing and context-aware gating to store and retrieve knowledge without materially raising the activated parameter count — capacity that lives in weights rather than cache, the mirror image of every previous chapter's move. Single-Pass mHC revises the V4 residual-mixing design into a kernel-fusable "Mega-mHC" pass that cuts activation memory traffic to (2n+2)d reads and writes — roughly half. Training infrastructure earned its own section: Attention Sharing Training manages "shadow indexers" (lightweight replicas of indexer state) to overlap communication and computation under pipeline parallelism; the Muon optimizer became head-wise (separate preconditioners per attention head); and the huge embedding tables train with Sinkhorn-Balanced Updates, a momentum-plus-balance scheme that shrank optimizer state while stabilizing the big tables. Post-training pushed further into agentic and multimodal data, and introduced Controllable Reasoning Effort: a scalar effort level (1–100) in the system prompt, trained with an exponential token-penalty reward, that lets one checkpoint behave as a quick assistant or a deep reasoner — up to 2.5× more output tokens at maximum effort.

14.6 — What 890 bytes buys

The benchmark table the launch circulated tells the story in three rows: Terminal-Bench 2.1: 90.6 for V4.1-Flash against 82.7 for V4-Pro (and Terminal-Bench 4.0: 31.2 against 7.0); DeepSWE v1.1: 74.2% of software-engineering issues resolved; a Codeforces rating of 3471 — with the smaller-and-cheaper model leading the larger one across the board. Pricing, at the September 10 effective date: input cache-hit $0.006/M (peak; $0.003 off-peak) against $0.30/M cache-miss — a 50× hit discount, the deepest in the industry and five times the 10× of 2024 — with output at $1.20/M peak. The 1M context and 384K maximum output round out a spec sheet aimed at exactly one workload: the long-horizon, tool-using, vision-fed agent whose transcript is 90% cache hit. The kernels shipped with the model: FlashMLA's V4.1 release covers prefill and decoding with FP8 or FP4 KV paths, and the official deployment note advertises direct engagement for 2,000-GPU-class clusters with storage backends — the cache is now small enough that SSD bandwidth is a first-class design constraint of the datacenter.

15. Compiled Evidence: Master Tables & Timeline

Every model, every number, one place — with the basis of each figure stated so you can check it.

This chapter is the reference file. It compresses Parts I–IV into four artifacts: the master cache table across the full 2023–2026 lineage, the traffic-and-price records that show what the cache bought, and the complete release timeline. Two reading rules apply throughout. First, "bytes per token" is only comparable when the basis matches: V1/V2/V3 figures are stored-cache arithmetic at bf16 (or fp8 where noted) from published configs; the V3.2 figure is stored bytes including the indexer cache; the V4 figures are DeepSeek's own percentages against V3.2 at 1M context; the V4.1 figure is the report's global in-HBM number including all pooling, sharing, and quantization effects. Second, read-size and storage-size are different quantities — DSA stores everything and reads little; V4.1 stores little in HBM, less on SSD, and reads a bounded amount. Where a number is DeepSeek's claim rather than independent arithmetic, the table says so.

15.1 — The master table

15.2 — Read traffic per decode step

15.3 — The price record

15.4 — The timeline

16. Field Guide: Rivals, Recipes & Sources

Where DeepSeek's designs sit among the alternatives, how to actually serve each generation, and the full source list.

16.1 — The mechanisms, compared

Three patterns in the table are worth naming. The industry's default (GQA) plateaued at ~8× while DeepSeek compounded 57× more on top of it; the labs that followed DeepSeek adopted its mechanisms rather than inventing parallel ones (Kimi K2 took MLA, Z.ai took DSA whole — including the layer-shared indexer refinement); and every DeepSeek mechanism degrades gracefully into a standard component (MLA decodes as SDPA, DSA adds a gather, CSA pools existing entries), which is precisely why the ecosystem could absorb each generation in days rather than quarters.

16.2 — Serving recipes, generation by generation

# ---- V3.2 (DSA): block size 64 enforced; fp8 native layout ----
vllm serve deepseek-ai/DeepSeek-V3.2-Exp --tensor-parallel-size 8 --trust-remote-code

# ---- V4-Flash (CSA+HCA): tagged vLLM release only ----
vllm serve deepseek-ai/DeepSeek-V4-Flash \
    --tensor-parallel-size 8 --enable-expert-parallel \
    --kv-cache-dtype fp8_ds_mla --trust-remote-code \
    --tokenizer-mode deepseek_v4 \
    --kv-transfer-config '{"kv_connector":"LMCacheMPConnector","kv_role":"kv_both"}'

# speculative decoding (native MTP head; validated with LMCache):
#   --speculative-config '{"method":"mtp","num_speculative_tokens":1}'

# start the LMCache movement-storage layer (100 GB L1, LRU eviction):
lmcache server --l1-size-gb 100 --eviction-policy LRU

# ---- API (V4.1-Flash): the cache-hit economics ----
curl https://api.deepseek.com/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"deepseek-flash",
       "messages":[{"role":"system","content":"...big static prefix..."},
                   {"role":"user","content":"..."}]}'
# prompt_cache_hit_tokens / prompt_cache_miss_tokens in usage:
# hits bill at $0.006/M (peak) - a 50x discount; keep prefixes stable,
# put volatile content LAST, and monitor hit ratio in the dashboard.

16.3 — Glossary

16.4 — Sources

Primary sources, in order of first citation. Where a claim in this paper is a quotation or a headline number, it comes from the first list; independent arithmetic and verification come from the second; context and color from the third.

One closing observation, offered as engineering rather than sentiment. In November 2023 the KV cache was a liability nobody designed around — a 389 KB-per-token tax inherited from the attention mechanism as given. Three years later it is the design center of the entire stack: the first sentence of DeepSeek's flagship technical report, the first number in its pricing table, the first constraint in its datacenter planning. The 890-byte cache is not the end of the story — the same report already treats SSD bandwidth and indexer cost as the next frontiers, and competitors are shipping their own variants of every mechanism here. But the pattern DeepSeek established is the durable one: find the multiplication nobody is attacking, and attack it for three consecutive years. The next multiplication is already being multiplied somewhere.