Context Management in Agent Loops

§ Abstract & How to Read This Paper

What this paper claims, what it deliberately does not claim, and how the evidence is organized.

An agent, in the definition Anthropic's applied AI team settled on in late 2025, is an LLM autonomously using tools in a loop. That definition has an uncomfortable corollary: the loop is a context-generating machine. Every planning step, every tool call, every raw JSON blob a tool returns, every reasoning trace a thinking model emits lands in the same finite window that must also hold the system prompt and the tool schemas. The window refills on every single call, because the model itself is stateless — each inference starts from zero, and only whatever the harness feeds back persists. Managing that feedback stream is now the difference between an agent that survives fifty turns and one that quietly degrades into a very confident, very expensive random walk.

This paper is written in September 2026, and it covers two intertwined stories. The first is the research landscape: the emergence of context rot as a measured phenomenon, the operating-system metaphor of MemGPT and its descendants (Letta, Mem0, A-MEM, MemOS, Zep), the retrieval rethink that replaced pre-fetching with just-in-time context and direct corpus interaction, the codification of the discipline by Anthropic, LangChain, and the practitioners at Manus, and the benchmark wars on LoCoMo, LongMemEval, BEAM, and DMR — including the uncomfortable fact that most head-to-head numbers in this field are published by the vendors being ranked. The second story is DeepSeek's, because no other lab has pushed context management as deeply into the model itself rather than the harness around it: Multi-head Latent Attention collapsed the KV cache by 93.3% in 2024, disk-based context caching made multi-turn agents dramatically cheaper the same year, DeepSeek Sparse Attention cut long-context attention to O(L·k) in 2025, V3.2 shipped explicit thinking-context-management rules for tool loops and published a controlled study of compaction strategies on BrowseComp, V4 grew the window to one million tokens with reasoning persistence across user turns, and V4.1-Flash (September 10, 2026) added a 196-billion-parameter Engram module that stores knowledge outside the attention stream and retrieves it with O(1) lookups — a memory system trained into the weights rather than bolted on as a retrieval tool.

How the paper is organized

Part I dissects the problem: what actually fills an agent-loop context, why quality degrades as the window fills (context rot, NoLiMa, lost-in-the-middle), and the economics of prefill-heavy loops, where prefix caching is not an optimization but the business model. Part II maps the research landscape from the 2023 foundations through the 2025–2026 memory-layer explosion and the codification of context engineering as a discipline. Part III is the DeepSeek deep-dive the title promises: version by version, from the dense 4K-context baseline of late 2023 to the Engram architecture of September 2026, with the exact context-management semantics of each release. Part IV assembles the practical payoff: a nine-pattern architecture catalog with trade-offs, a decision framework keyed to named constraints, and production recipes — including working code for the DeepSeek reasoning-retention loop that the V3.2 API contract makes mandatory.

Three audiences were kept in mind while writing. If you build agent harnesses, Parts I, II and IV are your material and Part III is the reference you will cite in design docs. If you train or serve models, Part III is the core and the benchmark chapters will save you from re-running settled arguments. If you are evaluating vendor claims about memory products, Chapter 8 is written specifically for you — it is a field guide to reading benchmark tables whose publishers have skin in the game.

1 Anatomy of an Agent-Loop Context

The loop, the stateless model, and the five freight cars that fill the window on every single call.

1.1 The loop, and the model that forgets

Strip away the frameworks and every agentic system in production today is the same eight lines of pseudocode: give the model a goal and a set of tools; let it emit either an answer or a tool call; execute the call in some environment; append the result; repeat. Anthropic's engineering essay on context engineering — the one that fixed the vocabulary for the whole industry in September 2025 — compresses this to a definition the field has largely converged on: agents are LLMs autonomously using tools in a loop. The loop is the easy part. The hard part is what the loop does to the model's context, because a transformer holds no state between invocations. Every pass through the loop re-sends everything the model has ever seen in the session: instructions, tool definitions, prior reasoning, prior observations. The model is stateless; the harness is the only memory it has.

This statelessness is the single most important architectural fact in agent engineering, and it was stated plainly by Louis-François Bouchard's team at Towards AI after running instrumented experiments on their open-source AI tutor in 2026: within a session you have context management; across sessions you have memory; and you cannot have good multi-session memory if a single session is already falling apart. The first half of that sentence is this Part I. The second half organizes the entire research field of Part II. Notice what the split implies: the "memory problem" that vendors sell products for is downstream of a more basic engineering discipline — deciding, at every call, which tokens earn their place in the window.

1.2 The five freight cars

Concretely, a production agent-loop call in 2026 typically carries five categories of content, and each has a different growth profile. The table below assembles a realistic accounting from the token budgets published in 2026 retrospectives by the Crux Digits engineering team and the Towards AI tutor experiments.

The surprising empirical finding, reported by the Towards AI team after instrumenting their tutor, is that the segment everyone instinctively trims — chat history — is usually the smallest offender. In their system, each call could pull up to 100K tokens of retrieved course material, and turns were landing around 200K input tokens; the retrieval payloads, not the conversation, dominated the input. Most advice about long agent sessions aims at trimming dialogue; in a retrieval-heavy agent, dialogue is rounding error. This single observation reshapes where engineering effort belongs: at the tool-output boundary and the retrieval filter, not at the transcript.

1.3 The skew: prefill-heavy, decode-light

Chatbot traffic is roughly symmetric — a human types a sentence, the model writes a paragraph. Agent traffic is not. Manus, whose engineering blog in July 2025 became the reference on this, measured their production agents at an average input-to-output token ratio of about 100:1. A typical task takes around 50 tool calls, each of which re-sends the entire accumulated trajectory so that the model can emit a few hundred tokens of structured function call. One consequence is immediate: the cost and latency of an agent is dominated by prefill — the recomputation of everything already known — not by generation. Another consequence is subtler and is the subject of Chapter 3: because each loop iteration shares a prefix with the previous one, the KV cache transforms agent economics from "pay for the whole window every turn" to "pay for the delta," and anything that breaks prefix sharing (a timestamp in the system prompt, a reordered JSON key, a mid-session tool-set change) silently re-inflates the bill by an order of magnitude.

1.4 What "context management" therefore means

Across every paper surveyed in Part II, the working definition that survives is negative: context management is the set of mechanisms that keep the effective context — the tokens the model actually re-reads — small, high-signal, and cache-stable, while the logical context — everything the session has ever produced — grows without bound. The mechanisms differ wildly in where they live: in the harness (compaction, note files, sub-agents), in the retrieval layer (just-in-time loading, corpus search), in the serving stack (prefix caching, KV compression), or in the model itself (attention sparsity, trained memory modules). The field's three-year history, traced in Part II, is essentially the migration of these mechanisms from the harness downward into the model. DeepSeek's late-2026 position — the focus of Part III — is the extreme end of that migration: at V4.1-Flash, every layer of the stack participates in context management, and the model itself now carries a dedicated 196B-parameter memory module.

2 Why Naive Context Fails

Context rot, the 32K cliff, lost-in-the-middle, and the attention budget: why bigger windows did not solve the problem.

2.1 Lost in the middle: the 2023 warning shot

The first systematic evidence that long contexts degrade non-uniformly predates the agent era entirely. Liu et al.'s Lost in the Middle study (2023) showed that a model's ability to use a document placed in the middle of a long context drops sharply relative to the same document placed at the beginning or end — a U-shaped positional performance curve. For agent builders the implication was blunt: the trajectory an agent appends turn after turn is, by construction, a pile of information drifting steadily toward the middle of the window, exactly where models read worst. Manus's engineers later turned this observation into an explicit design pattern — their agent rewrites its todo.md every few steps precisely to "recite" the plan into the end of the context, where attention is strongest (§7.6). The fix, in other words, was known before the disease was named.

2.2 Context rot: the 2025 measurement

By 2025, model vendors were shipping million-token windows, and it was tempting to declare the problem solved by brute force. Chroma Research's Context Rot study (July 2025) ended that temptation. The team tested 18 frontier models — including GPT-4.1, Claude 4, and Gemini 2.5 — on deliberately simple retrieval and replication tasks, and found that every single model degraded as input length grew, with accuracy cliffs arriving well before the advertised window limit. A 200K-token window could show serious accuracy loss by 50K tokens of input. Positional effects were severe: placing the relevant fact at positions 5–15 of a twenty-document context cost some models more than 30 accuracy points compared to placing it first or last.

Anthropic's engineering essay, published two months later, gave the phenomenon its working explanation and its budget metaphor. The transformer computes attention over n² pairwise token relationships; as context grows, "the model's ability to capture these pairwise relationships gets stretched thin." Models also train mostly on shorter sequences, leaving them under-adapted to context-wide dependencies. The result is "a performance gradient rather than a hard cliff" — and a finite attention budget that every additional token depletes. Context, in the essay's formulation, must be treated as a resource with diminishing marginal returns, like working memory in humans. The engineering consequence: the smallest set of high-signal tokens that maximizes the likelihood of the desired outcome is the optimization target — not the largest window you can afford.

2.3 NoLiMa: the 32K cliff, measured

The same quarter, Modarressi et al. published NoLiMa ("Long-Context Evaluation Beyond Literal Matching"), a benchmark that tests whether models can answer questions whose wording has minimal lexical overlap with the passage containing the answer — requiring latent, associative inference rather than literal string matching, which standard NIAH tests let models exploit. The result that agent engineers should memorize: across 13 models that all claim 128K+ context support, performance in short contexts (under 1K) was strong, but at 32K, eleven of the thirteen dropped below 50% of their own short-context baselines. Even GPT-4o — one of the top-performing exceptions — fell from an almost-perfect 99.3% baseline to 69.7%.

2.4 The compounding loop failure

Put the three findings together and the agent-loop failure mode emerges with uncomfortable clarity. An agent accumulates search results, tool outputs, and abandoned reasoning paths as it works. That accumulated noise degrades every subsequent step, not just the final answer — each new decision is made inside a context that is longer, more middle-heavy, and more associatively demanding than the last. The 2026 retrospectives converge on the same production observation: a ticket that reaches turn ten with an untrimmed transcript is simultaneously the most likely to hallucinate and the most expensive call of the conversation to run. Quality and cost fail together, on the same curve, because they share a cause. This is why the field stopped treating "context window size" as the headline specification and started treating context management as the discipline — and it is why every pattern in Part IV exists.

3 The Economics of the Loop

Prefix caching, the 10× cached-input discount, and why 2026's compaction debate is decided by cache arithmetic, not sentiment.

3.1 The cache is the business model

Because agent loops re-send an almost-identical prefix every turn, inference providers let you pay once for it. Anthropic's prompt caching prices Claude Sonnet's cached input tokens at $0.30 per million versus $3.00 uncached — a 10× difference, the exact figure Manus cites when it argues that the KV-cache hit rate is the single most important metric for a production-stage AI agent, directly driving both latency and cost. DeepSeek has operated the same bargain since August 2, 2024, when its Context Caching service launched as a disk-based, automatic feature: cache hits billed at $0.014 per million tokens against $0.14 for a miss — the same 90% discount, available two years before most of the industry's agent frameworks had a caching story. By September 2026, the V4.1-Flash release note states the stakes in one sentence: "Cache-hit charges often account for a large share of agent costs. Compressing the cache cuts those costs significantly."

The mechanics deserve precision, because the engineering rules fall straight out of them. An autoregressive transformer computes attention keys and values (the KV cache) for every token of the prefix; if the next request's prefix is byte-identical up to a point, the serving stack can skip recomputation and bill the reused span at the discounted rate. A single-token difference invalidates everything from that token onward. This yields the rule set Manus published: keep the prompt prefix stable (no second-precision timestamps at the top of the system prompt); make the context append-only; ensure deterministic serialization (many JSON libraries do not guarantee stable key ordering, which silently breaks caches); mark cache breakpoints explicitly where the provider requires them; and if self-hosting on vLLM, enable prefix caching and route requests to consistent workers with session identifiers.

3.2 The 2026 twist: caching inverted the compaction calculus

Compaction — summarizing a conversation as it nears the window limit — has been the reflex answer to long sessions since 2024. In August 2026, the Towards AI team published the measurement that broke the reflex: on their production tutor, evaluated with blind-graded memory probes, under modern prompt caching, keeping the full history beat every summarization strategy they tested on cost, latency, and memory recall simultaneously. The reason is cache arithmetic, not model quality: a summary rewrites the prefix, so you pay full price to recompute everything you just tried to save, and you lose the recall of the exact tokens you discarded. Their production defaults — which looked reasonable — scored 38% on memory probes and cost twice as much as doing nothing, while still producing answers a blind judge rated as good; their conclusion was that this combination is precisely why you cannot eyeball context strategy and must measure it.

What did pay, in the same experiments, was the cheap tier of compaction: capping every tool output at a stable size cut their cost per turn by 38% with no measurable loss in memory recall, because it shrinks the context without rewriting the prefix the cache depends on. The decision rule the team distilled: name your constraint before choosing a strategy — a window that genuinely does not fit, a cached input price above roughly $0.55 per million tokens, or measured quality rot. Each points to a different fix, and only the first one is "summarize."

3.3 The three-way trade

Every context-management decision in an agent loop trades three currencies against each other, and the research of 2025–2026 is best read as the systematic exploration of that trade space.

The rest of this paper is the story of how the field explored that trade space. Part II covers the research; Part III covers the one lab that pushed the frontier into the weights themselves; Part IV assembles the patterns into something you can deploy this week.

4 Foundations & the Operating-System Metaphor

Memory streams, reflection, skill libraries — and MemGPT's 2023 bet that an LLM could manage its own memory like an OS manages RAM.

4.1 The pre-history: cognitive architectures on GPT-3.5

Before "agent" was an infrastructure category, three 2023 systems established that what you put around the model matters as much as the model. Generative Agents (Park et al., 2023) gave 25 LLM-powered agents a persistent memory stream and a retrieval score computed as a weighted blend of recency, importance, and relevance — plus a reflection mechanism that periodically distilled the stream into higher-level observations. That scoring formula is still the skeleton inside most "agentic memory" products sold in 2026. Reflexion (Shinn et al., 2023) demonstrated verbal self-improvement: an agent that stores its own failure analyses as episodic memory and re-reads them on retry outperforms the same agent without that verbal loop — an early proof that what an agent writes down about its own trajectory changes its future behavior. Voyager (Wang et al., 2023) externalized procedural memory as a growing skill library of verified code, showing that a file-shaped memory an agent both writes and retrieves beats any amount of in-context rehearsal for long-horizon competence. All three prefigure the 2026 consensus patterns: write state down, score it, retrieve it just in time, and reflect on it offline.

4.2 MemGPT: virtual context management (October 2023)

The paper that named the field arrived from UC Berkeley in October 2023. Packer et al.'s MemGPT: Towards LLMs as Operating Systems proposed virtual context management: treat the fixed context window as a limited quantity of main memory, and manage a much larger external context (conversational archives, raw documents) through explicit paging operations — evictions, fetches, and writes that the LLM itself triggers by calling functions. The OS analogy is precise and was meant to be: just as an operating system gives each process the illusion of unlimited RAM by paging to disk, MemGPT gives the model the illusion of unlimited context by paging between the window and external stores, with the model acting as its own memory manager via self-editing function calls.

Two design commitments in the original system matter for everything downstream. First, memory is structured by role, not one undifferentiated log — the system splits a persistent in-window region (core memory, including editable "blocks" like persona and human descriptions) from paged regions (recall storage for conversation history, archival storage for documents). Second, memory operations are agentic: rather than a fixed summarizer running on a timer, the model invokes memory-editing functions when it judges them necessary. Both commitments survive intact in Letta — the company built by the MemGPT team — and in every system that copied the design. The 2025–2026 memory-layer products of Chapter 5 are, to first order, competing answers to one question MemGPT posed: who decides what gets paged — the model, the pipeline, or the graph?

4.3 Letta: blocks, filesystems, and sleep-time compute

Letta, the continuation of MemGPT by its original team, contributed three developments that anchor the "agentic memory" wing of the field. Memory blocks (2025) formalized the in-window core memory as typed, editable, shareable segments — a persona block, a project block, a human block — that the agent rewrites through tool calls and that multiple agents can share, making memory a first-class interface rather than a prompt appendix. Sleep-time compute (arXiv 2504.13171, April 2025) introduced the idea that an agent can think offline about its context between queries: anticipating likely future questions and reorganizing its own memory during idle time, converting wall-clock that used to be wasted into retrieval quality at query time. It is the direct descendant of Generative Agents' reflection, industrialized.

The third contribution is the 2025 benchmark counter-attack. In Benchmarking AI Agent Memory: Is a Filesystem All You Need? (August 12, 2025), Letta attached LoCoMo conversation histories to agents as plain files navigable with grep, search_files, open, and close tools — no memory product at all — and scored 74.0% on LoCoMo with GPT-4o-mini, above the 68.5% Letta cites for the top graph variant in Mem0's own published results. Their conclusion cuts the legs out from under half the memory market: agents are highly effective at filesystem tools (because those are in their training data), so the binding constraint is the agent's ability to manage context, not the sophistication of the retrieval mechanism. Comparing frameworks-and-tools mixes is apples-to-oranges; the Letta Memory Benchmark therefore holds the harness constant and varies the model. The blog also documents a reproducibility dispute worth remembering when reading vendor tables: the MemGPT team could not reconstruct how Mem0's paper produced its MemGPT baseline numbers, and Mem0 did not respond to clarification requests — a preview of Chapter 8's broader lesson.

5 The Memory-Layer Explosion

Mem0, A-MEM, MemOS, Zep: four 2025 answers to MemGPT's question, and what each actually demonstrated.

5.1 Mem0: extraction pipelines as a managed service

Mem0 (Chhikara et al., arXiv 2504.19413, published at ECAI 2025) is the industrial wing of the memory movement. Its architecture is a two-phase pipeline rather than an agentic loop: an extraction phase that pulls salient facts from each conversational turn, and an update phase that resolves each new fact against the store with four operations — ADD, UPDATE, DELETE, and NOOP. The paper's headline results on LoCoMo: a 26% relative improvement in the LLM-as-a-Judge metric over OpenAI Memory; a graph-memory variant scoring around 2% higher than the base configuration; 91% lower p95 latency and more than 90% token-cost savings versus a full-context baseline that dumps the whole history into the window. The token-savings figure is the strategically important one: it is the same economics as Chapter 3's caching argument, achieved by not re-sending history at all.

In April 2026, Mem0's engineering team published a State of AI Agent Memory report introducing a redesigned algorithm — single-pass hierarchical extraction with ADD-only storage, plus multi-signal retrieval that fuses semantic, keyword, and entity matching in parallel — and new numbers: 92.5 on LoCoMo at ~6,956 tokens per query, 94.4 on LongMemEval at ~6,787, 64.1 on BEAM (1M scale) and 48.6 on BEAM (10M scale), with the largest category gains on temporal reasoning (+29.6 points) and multi-hop (+23.1). The report is also a useful market document: Gartner's projection that 40% of enterprise applications will integrate task-specific AI agents by end of 2026 (from under 5% in 2025) sits alongside McKinsey's more measured 2025 survey finding (23% of organizations scaling agentic AI, 39% experimenting), and its own integration census — 21 agent frameworks, 20 vector-store backends — maps the ecosystem's fragmentation better than any academic survey could. Its stated open problems (cross-session identity, temporal abstraction at scale, memory staleness) are reproduced in Chapter 16's outlook.

5.2 A-MEM: Zettelkasten notebooks that organize themselves

A-MEM (Xu et al., arXiv 2502.12110, February 2025, NeurIPS 2025) answers MemGPT's question with the model, fully. Its design borrows the Zettelkasten note-taking method: every memory is a structured note (content, keywords, tags, a contextual description generated at insertion time); an evolution mechanism then dynamically generates links between notes and, when a new note arrives, may trigger the re-organization of existing neighbors — memory that restructures itself as evidence accumulates, rather than a store that merely grows. The system demonstrated that dynamically organized, self-linking memories outperform static alternatives on long-horizon conversational QA, and its construction–link–evolve pipeline became the template for the "agentic memory" product category that followed it. Conceptually, A-MEM is the retrieval graph the Mem0 graph variant buys, minus the pipeline — the agent is the pipeline.

5.3 MemOS: memory as a schedulable resource

MemOS (Li et al., arXiv 2507.03724, 2025) pushes the operating-system metaphor past paging into scheduling. Its core abstraction, the MemCube, unifies three kinds of memory under one management layer: plaintext memory (facts and text, as every other system stores), activation memory (KV caches retained and reused across calls), and parametric model memory — with the system deciding transformations between them, scheduling when each tier is consulted, and treating memory as a first-class system resource with its own governance. The ambition is to make "memory management" a systems discipline with the same rigor as memory management in an OS: admission policies, tiering, and lifecycle management rather than a pile of vectors. The KV-cache-as-memory move is the one with teeth — it is the same insight DeepSeek industrialized at the serving layer (Part III), and that production stacks like LMCache pursue for open-weight models.

5.4 Zep: the temporal knowledge graph

Zep (arXiv 2501.13956, January 2025) rejects the vector store and the flat note alike, organizing memory as a temporally-aware knowledge graph built by its engine, Graphiti. Episodes, entities, and relations are extracted from conversations and business data as they stream in; each fact carries validity intervals, so the graph knows when something was true — the mechanism NoLiMa showed literal-matching systems lack, applied to memory rather than windows. The paper's reported results: 94.8% accuracy on the DMR benchmark, beating MemGPT's 93.4% (a benchmark the MemGPT team itself created), and on LongMemEval, accuracy improvements of up to 18.5% with response latency reduced by 90% versus full-context baselines. Zep's team also published the field's most aggressive counter-benchmarking: a May 2025 blog arguing Mem0's SOTA claim didn't hold under re-testing, reporting Zep outperforming Mem0 by 24% on DMR — while Mem0's 2026 report returns fire with LoCoMo configurations where Zep scores 80.32–83% against Mem0's 92.5. Chapter 8 treats this crossfire as data, not noise.

6 Retrieval Rethought

From pre-fetching everything to just-in-time loading, direct corpus interaction, and quarantined sub-agent contexts.

6.1 Just-in-time context: the Anthropic doctrine

The dominant retrieval pattern of the RAG era was pre-inference: embed everything, fetch the top-k chunks, stuff them into the prompt before the model thinks. Anthropic's September 2025 essay documented the shift the field was already making: agents that maintain lightweight identifiers — file paths, stored queries, web links — and pull the referenced data into context at runtime, only when needed. Claude Code is the flagship example: it performs complex analysis over large databases by writing targeted queries, storing results, and using head and tail over files — never loading the full data objects into the window. The essay frames this as cognitively honest ("we don't memorize corpuses; we build filing systems") and operationally superior: the metadata of a reference (a file's name, its directory, its timestamp) carries signal that a stuffed chunk does not, and the agent's exploration becomes progressively disclosing — each read informs the next query.

6.2 Direct corpus interaction: agents grep better than embeddings rank

The 2026 research result that formalized the pattern is Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction (arXiv 2605.05242, May 2026). Its proposal, DCI, gives the language agent an interface to query raw text directly — search, filter, and read over the corpus itself — instead of consuming a pre-ranked embedding shortlist. The empirical finding: agents using DCI outperform traditional similarity-based retrieval pipelines on agentic search tasks, because the agent can reformulate its queries mid-search (the same competence Letta's filesystem experiment isolated) while embedding pipelines rank once, blind to what the agent has since learned. Combined with Letta's 74.0% filesystem result, the 2026 picture is coherent: retrieval is no longer a pre-processing stage but an agentic behavior, and the corpus interface — files and grep — is the one models already know from their coding post-training.

6.3 Sub-agent architectures: isolation as a context strategy

The third retrieval-side pattern is not retrieval at all: it is context quarantine. Anthropic's essay describes the orchestrator-worker design in which specialized sub-agents take focused questions into clean windows, explore exhaustively (tens of thousands of tokens each), and return condensed summaries of 1,000–2,000 tokens to the lead agent, which thereby never sees the noise. Their multi-agent research system is the reference deployment. The warning label comes from the practitioner side: Philipp Schmid's December 2025 retrospective names the failure mode context pollution — when sub-agents share one context, you pay the KV-cache penalty anyway and the shared noise confuses every model reading it. Isolation only pays when the boundary is real: fresh window per sub-agent, distilled handoff at the seam. DeepSeek's V4 harness takes the same discipline to training time, with the DSec sandbox platform evaluating agentic rollouts in isolated execution environments (§12.5).

6.4 The four levers: LangChain's consolidation

LangChain's engineering team consolidated the whole design space into four verbs that now organize most production agent frameworks: Write (persist context outside the window — scratchpads, memory files, running plans), Select (pull the right information in at the right moment — tools, retrieval, file reads), Compress (reduce what is already in the window — summarize resolved turns, trim tool outputs), and Isolate (split work across separate windows — sub-agents, sandboxed steps). Their January 2026 "Context Management for Deep Agents" post describes the Deep Agents SDK implementing exactly these through offloading, summarization, and filesystem abstraction, and LangGraph's checkpointed state graphs became the 2026 default substrate for auditable long-running agents because the four operations map onto first-class graph constructs. The four-lever frame is the taxonomy Part IV's pattern catalog is organized around — it is descriptive, not competitive, and nothing in the catalog falls outside it.

7 Context Engineering Grows Up

Karpathy names it, Anthropic codifies it, ACE makes it self-improving, and Manus ships the production field manual.

7.1 From prompt engineering to context engineering

On June 25, 2025, Andrej Karpathy posted the tweet that renamed the discipline: "+1 for 'context engineering' over 'prompt engineering'… the delicate art and science of filling the context window with just the right information for the next step." The renaming mattered because it relocated the engineering problem. Prompt engineering optimizes one static instruction; context engineering manages the full, changing set of information an agent sees at every step of a multi-turn task — system prompt, tools, examples, memory, history, runtime data — as one curated, dynamically assembled artifact. Anthropic's essay adopted the frame in September 2025 and gave it the working definition used throughout this paper: the set of strategies for curating and maintaining the optimal set of tokens during inference, iteratively, on every call.

7.2 Compaction, done carefully

Anthropic's essay treats compaction as the first lever of long-horizon coherence: as a conversation nears the window limit, summarize its contents and reinitiate a new context from the summary. The Claude Code recipe is concrete enough to implement verbatim: pass the message history to the model to summarize, preserving architectural decisions, unresolved bugs, and implementation details while discarding redundant tool outputs and messages, then continue with the compressed context plus the five most recently accessed files. The essay is equally explicit about the risk: overly aggressive compaction loses "subtle but critical context whose importance only becomes apparent later," and the recommended tuning order is recall-first (capture everything relevant), then precision (eliminate the superfluous). The lightest-touch form — tool result clearing, deleting the raw payload of deep-history tool calls whose fact has already been absorbed — shipped as a platform feature on the Claude Developer Platform in September 2025 (with the documented caveat that cleared spans invalidate the corresponding cached prefix). DeepSeek's V3.2 ran the same play at the model level, discarding reasoning traces at user-turn boundaries while preserving tool-call history — and then benchmarked the alternatives on BrowseComp; both stories appear in Chapter 11.

7.3 Structured note-taking: the NOTES.md pattern

The essay's second pattern is structured note-taking, or agentic memory: the agent regularly writes notes to files outside the window and pulls them back in later — a to-do list in Claude Code, a NOTES.md in a custom agent — "tracking progress across complex tasks, maintaining critical context and dependencies that would otherwise be lost across dozens of tool calls." The demonstration that convinced the skeptics is Claude playing Pokémon: with no prescribed memory format, the agent maintained precise tallies over thousands of game steps ("for the last 1,234 steps I've been training my Pokémon in Route 1, Pikachu has gained 8 levels toward the target of 10"), drew maps of explored regions, and recorded combat strategies; after context resets it re-read its own notes and resumed multi-hour objectives. The production version is Anthropic's memory tool (public beta since the Sonnet 4.5 launch): a file-based system that lets agents build knowledge bases over time and reference previous work without keeping it all in context.

7.4 Agentic Context Engineering: the playbook that rewrites itself

The most conceptually aggressive 2025 research result is ACE (Zhang et al., arXiv 2510.04618, October 2025; Stanford, SambaNova, UC Berkeley). ACE splits the context-improvement job across three roles: a Generator produces task trajectories; a Reflector distills concrete lessons from what worked and what failed; and a Curator folds those lessons back into a structured, evolving context playbook — the system prompt stops being a static artifact written once by an engineer and becomes a self-improving artifact maintained by the system. On their evaluations, ACE-managed contexts beat strong static baselines without a single gradient update: +10.6% on agent tasks and +8.6% on financial-analysis tasks. The model did not get smarter; its context did. ACE is the bridge between this chapter and Part III: it treats context itself as trainable state — the same philosophical move DeepSeek made when it trained a 196B-parameter memory module into the weights (§13).

7.5 Model-native features: the provider layer joins in

By late 2025, context management had stopped being purely a harness discipline: the model APIs themselves began shipping native mechanisms. Anthropic's September 2025 release wave brought the context-editing capability set — tool result clearing and thinking-block clearing with cache-interaction semantics documented in the platform's context-window guide — alongside the memory tool. The platform docs formalized the full accounting: everything in the request counts toward the window, including tool definitions and thinking tokens, with cache reads split out in the usage field. Claude's chat interface manages the window on a rolling first-in-first-out basis — the interface-level version of truncation. The significance for architects: the boundary between "harness pattern" and "model feature" is dissolving, and procurement decisions now include which native context primitives a provider exposes alongside price and quality. DeepSeek's reasoning-retention contract (§11.3) is exactly such a primitive — an API-level context policy with an error code attached.

7.6 The Manus field manual: seven rules from production

The single most-cited practitioner document of the era is the Manus engineering blog, "Context Engineering for AI Agents: Lessons from Building Manus" (Yichao 'Peak' Ji, July 18, 2025). Its rules deserve quoting in structure, because each one encodes a failure their team hit in production:

• Design around the KV-cache. The hit rate is the number-one production metric; keep the prefix stable, the context append-only, serialization deterministic; set explicit cache breakpoints; use consistent session routing. (The 100:1 ratio and the 10× cached-input price from Chapter 3 are this rule's economic foundation.)
• Mask, don't remove. Dynamically adding or removing tools mid-loop invalidates the cache from the tool-definition position and strands history that references now-undefined tools, producing schema violations and hallucinated calls. Instead, Manus constrains the action space at decode time — a context-aware state machine that masks tool logits via response prefill, exploiting the three standard function-calling modes (auto / required / specified), plus naming conventions (browser_*, shell_*) that make group constraints mechanical.
• Use the file system as context. Observations can be huge; performance degrades with length; long inputs are expensive even cached. So: "the file system as the ultimate context — unlimited in size, persistent by nature, and directly operable by the agent itself," with compression always restorable (drop the web page, keep the URL; drop the document, keep the path).
• Manipulate attention through recitation. The todo.md behavior is deliberate: by rewriting the task list every few steps, the agent "recites" its objectives into the end of the context — recent positions, where attention is strongest — countering lost-in-the-middle and goal drift across ~50-tool-call trajectories.
• Keep the wrong stuff in. Erasing failures removes the evidence the model needs to update its beliefs away from the failed action; error recovery is treated as a marker of genuinely agentic behavior. (Contrast the common instinct to hide errors and retry "clean.")
• Don't get few-shotted. A context full of near-identical action–observation pairs teaches the model to imitate the pattern even when it is no longer optimal; Manus injects structured variation (serialization templates, phrasing, ordering) to break the mimicry.
• Expect to rebuild. Their agent framework was rebuilt four times as better context shapes emerged — their name for the process, "Stochastic Graduate Descent," is the field's most honest methodology statement.

8 The Benchmark Wars

LoCoMo, LongMemEval, BEAM, DMR: what each benchmark actually measures, and how to read vendor tables whose publishers are also contestants.

8.1 What the four benchmarks measure

The evaluation stack that consolidated around these benchmarks combines five dimensions — BLEU, F1, an LLM-judge score, tokens consumed per query, and wall-clock latency — precisely so that a system cannot win on accuracy while quietly spending 26,000 tokens per question. The discipline of reporting token consumption per query is the benchmark suite's most useful product: it is the Chapter 3 economics, instrumented.

8.2 The crossfire, documented

Here is the same field as reported by its contestants, side by side, with sources named. The numbers are all real; the configurations are not comparable; that is the point.

8.3 How to read a memory benchmark in 2026

The wars teach a reading protocol that costs nothing and prevents most expensive mistakes. Check the model under the harness. LoCoMo scores move tens of points with the driver model (Letta's 74.0 was deliberately run on GPT-4o-mini to match a disputed baseline; Mem0's 92.5 runs on a frontier stack). Check the judge. LLM-as-a-Judge scores inherit the judge's biases, and different papers use different judges. Check the units. Mem0's 2025 paper reported tokens per conversation (~26,000 for full-context) while the 2026 algorithm reports tokens per retrieval call (~6,956) — "different units measuring the same underlying efficiency," in their own footnote. Check who ran it. The three most-cited head-to-head tables in the field were published by Mem0, Zep, and Letta, each of which sells the winner. Prefer dynamic and holistic evaluation. Letta's Memory Benchmark creates memory interactions on the fly (defeating training-set contamination and testing management, not just retrieval), and Terminal-Bench-style task benchmarks measure memory as it actually manifests: an agent holding task state across a long-running job. And when a vendor's number matters to a decision, re-run it — Mem0 open-sourced its benchmark suite (github.com/mem0ai/memory-benchmarks) for exactly this reason, and the Towards AI team's 2026 finding that their own production defaults looked fine and measured badly is the cautionary tale for skipping the step.

9 Baseline Era: MLA & Context Caching

2023–2024: the dense 4K-context starting point, the 93.3% KV compression that made long context affordable, and the disk cache that made agent loops cheap.

9.1 The 2023 baseline: no context management at all

DeepSeek LLM 7B and 67B (November 2023) are the honest baseline for this history: dense transformers with standard attention (MHA in the 7B, grouped-query attention in the 67B), a 4K-token context window, and not a single mechanism aimed at context management. Every problem documented in Part I existed here in its purest form: a multi-turn tool loop hit the window after a handful of observations; attention cost grew quadratically with the trajectory; the only available strategies were truncation and hope. The line matters because the rest of this Part is measured against it: the V4.1-Flash release materials compute their KV-cache footprint against a "~390,000 bytes per token" figure that corresponds to this era's full-precision multi-head attention across a deep stack — a 438× gap that closed in under three years (§13.4).

9.2 V2 (May 2024): MLA makes the KV cache 93.3% smaller

DeepSeek-V2 introduced Multi-head Latent Attention (MLA), the first architectural bet that context economics belong inside the model. MLA jointly compresses the keys and values of every attention head into a small latent vector cached per token, decompressing them on the fly during attention; the RoPE positions ride in a small decoupled stream. The paper's own headline numbers: a 93.3% reduction in KV-cache size relative to the 67B dense baseline's attention, a 5.76× maximum decoding throughput improvement under cache-bound serving, and 42.5% lower training cost for comparable generation quality. In V3's 61-layer configuration, the per-token KV footprint works out to roughly 70 KB in bf16, against ~500 KB for an 8-group GQA and ~4 MB for full MHA — the compression that made "re-prefill the whole trajectory every turn" survivable at all. The window was already there — V2 itself supported 128K context, YaRN-extended from the 4K pre-training length with NIAH results reported clean across the range — so MLA's decisive contribution was not the window but the cost curve of filling it. Every model in the V3 line inherited the mechanism (the V4 generation replaced it with its hybrid compressed-attention stack, §12), and every serving stack that runs open-weight agents at scale — vLLM, SGLang, LMCache offloading — is built around MLA-shaped caches.

9.3 Context Caching (August 2024): the agent-loop business model, shipped

On August 2, 2024, DeepSeek's API launched Context Caching: a disk-based, automatic cache of KV states for previously seen prefixes, billed at $0.014 per million tokens on hits against $0.14 on misses — the 90% discount that Chapter 3 established as the economic core of agentic inference. The engineering choice to make it automatic is worth pausing on: with no cache breakpoints to manage and no manual invalidation API, every multi-turn conversation and every agent loop with a stable prefix benefits by default, which is precisely the population Manus later identified as the ones whose cost is decided by hit rate. Two years on, the pricing page has only sharpened the bargain — V4.1-Flash cache hits are $0.006 per million tokens at peak — and the release note for V4.1-Flash explains why the discount keeps deepening: cache-hit charges often account for a large share of agent costs, and compressing the cache cuts those costs. The compression is the next chapter.

9.4 V3 and R1: the window grows, and reasoning makes context heavier

DeepSeek-V3 (December 2024) kept the 128K context and added Multi-Token Prediction — which sped decode (1.8× tokens-per-second with the MTP module as a speculative decoder at an 85–90% acceptance rate) but left context management to the harness, as before. R1 (January 2025) then changed the problem's shape: a reasoning model emits long chain-of-thought before every answer, and in agent loops that reasoning lands in the window alongside tool outputs. The R1-era serving contract was the blunt one: reasoning content was discarded at each new user message. The V3.2 paper later named the cost of that policy precisely — "the model is forced to redundantly re-reason through the entire problem for each subsequent tool call" (§11.1) — making R1's contract the explicit baseline that DeepSeek's next two releases would be engineered against. The lesson generalizes: reasoning traces are context freight, and a context-management story that ignores them is already obsolete.

10 V3.2-Exp: DSA, the Cheap Long Context

September 2025: the lightning indexer, O(L·k) attention, a 5× decode-cache cut at 131K, and a 50%+ price drop — the substrate every later context feature stands on.

10.1 The attention tax, and how DSA removes it

The V3.2 release line began with DeepSeek-V3.2-Exp on September 29, 2025, carrying DeepSeek Sparse Attention (DSA). Vanilla attention scores every query against every key — O(L²) in sequence length — which is exactly the tax an agent's growing trajectory pays on every turn. DSA adds a lightning indexer: a tiny, FP8-quantized scoring network (64 shared heads of 128 dimensions, ReLU-activated) that computes a cheap relevance score between every query and every key, after which each query attends only to its top-k keys (k = 2048) — cutting the main model's attention complexity from O(L²) to O(L·k), with k independent of context length. The indexer itself still scans the sequence, but at a fraction of the main model's per-pair cost, and it runs in a masked-MHA shortcut regime for short prompts where full attention is cheaper than selection.

10.2 The cache arithmetic, in bytes

For serving, DSA split the KV budget in two: the compressed MLA entries (which the paper's deployment numbers put at ~656 bytes per token per layer in the production stack: 512 bytes of FP8 latent plus scales, and 128 bytes of bf16 RoPE keys) and the indexer scores, which are small enough to store at ~132 bytes per token per layer (128 bytes FP8 + scale). Because only the top-k entries are needed at decode, the working set at a 131K-token context shrinks from ~5.2 GB to ~1.1 GB per batch element in the deployment's accounting — a ~5× reduction in decode-time cache traffic, delivered by DeepGEMM's fp8_mqa_logits indexer kernels and a sparse FlashMLA attention kernel. Independent long-context evaluations after release (Artificial Analysis' long-context reasoning suite) put V3.2-Exp four points above V3.1-Terminus in reasoning mode — evidence that the sparsity did not trade quality for the efficiency. The pricing followed the cost: input and output prices dropped by more than 50% (output to $0.28 per million tokens), the first time a DeepSeek architecture release was explicitly priced as an agent-infrastructure event.

11 V3.2: Thinking Context Management

December 2025: the first model with an explicit context-management policy for tool loops, an API contract with a 400 error attached, and a controlled study of compaction strategies on BrowseComp.

11.1 The policy: retain reasoning across tool calls, discard on user turns

DeepSeek-V3.2 (December 1, 2025; arXiv 2512.02556) is, by its own framing, a "reasoning-first model built for agents," and its signature context-management contribution is Section 3.2.1, Thinking Context Management. The reasoning is stated as a rejection of the R1 baseline: discarding reasoning content whenever a new round of messages arrives "results in significant token inefficiency," forcing the model to "redundantly re-reason through the entire problem for each subsequent tool call." The replacement policy, tuned specifically for tool-calling scenarios, has three rules:

• Rule 1 — tool outputs do not wipe reasoning. Historical reasoning content is discarded only when a new user message enters the conversation. If only tool-related messages (e.g., tool outputs) are appended, the reasoning content is retained throughout the interaction.
• Rule 2 — user turns are the reset boundary. When reasoning traces are removed (at a genuine user turn), the removal is wholesale: the model re-reasons for the new request.
• Rule 3 — tool history outlives reasoning. When reasoning traces are removed, the history of tool calls and their results remains preserved in the context.

The paper is candid about the boundary condition: agent frameworks that simulate tool interactions via user messages — it names Roo Code and Terminus — may not trigger the tool-calling path and therefore "may not fully benefit from our enhanced reasoning persistence"; for such architectures DeepSeek explicitly recommends non-thinking models. This is a documentable, testable semantic, and it is the first time a frontier lab shipped a context-management policy as a model behavior rather than a framework convention.

11.2 Training the behavior in

The policy is not a post-hoc serving trick; it is trained. V3.2's post-training unified reasoning and tool-use in single trajectories, starting from a cold-start that prompted a V3-style model to interleave multiple tool calls inside its reasoning, then scaling through reinforcement learning on synthesized agentic tasks: over 1,800 distinct environments and 85,000 complex instructions, spanning search, code-engineering, and code-interpretation tasks. The release notes headline it as "our first model to integrate thinking directly into tool-use," supporting tool-use in both thinking and non-thinking modes. The high-compute sibling of the release, V3.2-Speciale, deliberately omitted tool use (API-only, at the time) to push reasoning benchmarks to gold-medal level — the split itself illustrating that agentic context behavior and pure reasoning length were being optimized as separate axes. On the agentic side of that split, V3.2's reported scores include Terminal-Bench 2.0 at 46.4 (against Claude-4.5-Sonnet's 42.8, GPT-5's 35.2, and Gemini-3.0-Pro's 54.2 in the same table), SWE-Verified at 73.1, and τ²-Bench at 80.3 — with the paper's framing that V3.2 "significantly advances the agentic capabilities of open models" while remaining "a highly cost-efficient alternative in agent scenarios."

11.3 The API contract: reasoning_content is mandatory in tool loops

The same policy is enforced at the API surface, in DeepSeek's thinking-mode guide. In thinking mode the model's chain-of-thought is returned as reasoning_content, a sibling of content. Whether it persists into the next request's context depends on the request's shape:

• With the tools parameter present (agent loops): the reasoning_content of all previous turns must be passed back — even for turns where the model performed no tool call. If the harness drops it, the API returns a 400 error.
• Without tools (plain chat): the reasoning content of prior turns is ignored even if passed — the chat-side discard rule, applied server-side.

This is the cleanest instance in the industry of context management as a contractual API primitive: the retention policy from Section 3.2.1, expressed as request-validation semantics with an error code. Chapter 16's recipe implements the compliant loop.

11.4 The experiment: compaction strategies under a compute budget

Section 4.4 of the V3.2 paper — Context Management of Search Agent — is the part agent architects should read twice, because it is the one controlled, published comparison of harness-level context strategies from a frontier lab. The motivation: even at 128K, agentic search workflows "frequently encounter maximum length limitations that prematurely truncate the reasoning process," capping test-time compute. The intervention, triggered when token usage exceeds 80% of the context window:

• Summary — summarize the overflowed trajectory and re-initiate the rollout. Measured effect: average steps extended to 364, with performance improving to up to 60.2 on BrowseComp — effective but token-expensive relative to its gains.
• Discard-75% — drop the first 75% of the tool-call history to free space.
• Discard-all — reset the context by discarding all previous tool-call history; the paper notes the similarity to the new-context tool Anthropic shipped in 2025. Despite its brutality, it "performs well in both efficiency and scalability," reaching 67.6 on BrowseComp with significantly fewer steps than the parallel baseline.
• Parallel-fewest-step (the comparison baseline) — sample N independent trajectories and keep the one that finishes in the fewest steps.

Two results deserve emphasis. First, the headline: DeepSeek-V3.2's BrowseComp moves from 51.4 to 67.6 when context management is applied (the table's asterisked number; the same table lists Kimi-K2-Thinking with 60.2 under its own context management). Context management is not an operational nicety — on this benchmark it is worth +16.2 points, a bigger jump than most model-generation improvements deliver. Second, the counter-intuitive ordering: the crudest strategy beat the sophisticated one. Discard-all (67.6) outscored Summary (60.2) while using far fewer tokens, "comparable to parallel scaling while using significantly fewer steps." The paper's own conclusion frames the design space: test-time compute can scale serially through context management or in parallel, and "finding the optimal combination of serial and parallel scaling… remains a crucial direction for future work."

12 V4: A Million Tokens Built for Agent Loops

April 2026: the preview that reframed context as the product — reasoning persistence across user turns, CSA+HCA hybrid attention, Quick Instruction's cache-native auxiliary tasks, and a 500-step/512K agent harness.

12.1 The reframing: "a million-token context that agents can actually use"

DeepSeek-V4 arrived as a preview on April 24, 2026 (technical report arXiv 2606.19348) in two configurations — V4-Pro at 1.6T total parameters (49B activated) and V4-Flash at 284B (13B activated) — and with a framing that is itself a data point for this paper's thesis: the Hugging Face announcement titled it "a million-token context that agents can actually use." The efficiency claims that justify the phrasing, at one million tokens of context: V4-Pro requires 27% of the single-token inference FLOPs and 10% of the KV cache that DeepSeek-V3.2 would need; V4-Flash runs at 10% and 7% respectively. The report's comparison table scores the V4-Pro-Max configuration at 83.5 MMR on Long-MRCR-1M and 62.0 ACC on CorpusQA-1M — trailing Opus-4.6 (92.9 and 71.7) but demonstrating retrieval that still functions at the 1M scale: the quality-side evidence that the million tokens are usable, not nominal. The architecture behind the numbers is a hybrid attention stack (Compressed Sparse Attention interleaved with a Heavily Compressed Attention) whose details are documented in the V4 report's architecture chapter; for this paper, the relevant point is what the efficiency buys: enough headroom to stop discarding context.

12.2 Interleaved thinking, refined for agents

Chapter 11 documented V3.2's policy: reasoning persists across tool calls but is flushed at user-message boundaries — which, the V4 report notes, "still caused unnecessary token waste in complex agentic workflows: each new user turn would flush all accumulated reasoning content, forcing the model to reconstruct its problem-solving state from scratch." V4's refinement splits the semantics by scenario:

• Tool-calling scenarios: all reasoning content is fully preserved throughout the entire conversation — including across user-message boundaries — maintaining "a coherent, cumulative chain of thought over long-horizon agent tasks."
• General conversational scenarios: the V3.2 rule stands (reasoning from prior turns is dropped when a new user message arrives), "keeping the context concise for settings where persistent reasoning traces provide limited benefit."

The caveat survives verbatim: frameworks that fake tool calls as user messages (Terminus is the named example again) do not trigger the tool-calling path and get the conversational semantics instead — and DeepSeek continues to recommend non-thinking models for those architectures. Read together, Chapters 11 and 12 give a lab-internal, two-step evolution of the same policy, each step justified by measured token waste in the previous one: R1's flush-everything → V3.2's flush-at-user-turns → V4's keep-everything-when-tools-are-real. No other frontier lab has published this reasoning trail.

12.3 The agent harness, stated in the report

DeepSeek's agent evaluations in the V4 report use an internally developed framework whose configuration is itself a reference design: a minimal tool set — a bash tool and a file-edit tool; a maximum of 500 interaction steps; a maximum context of 512K tokens (half the model's window). The reported results include Terminal-Bench 2.0 at 67.9 for the V4-Pro-Max configuration (the report separately cites approximately 72.0 on the Verified subset for V4-Pro) and SWE-Verified at 80.6, with search-agent tasks (BrowseComp, HLE with tools) run in a separate websearch-plus-Python harness — and, notably, BrowseComp evaluated with "the same discard-all context management strategy as DeepSeek-V3.2": the §11.4 finding carried forward into the next generation. Behind the evaluations sits real infrastructure: the report describes DSec (DeepSeek Elastic Compute), a production sandbox platform of three Rust services — API gateway, per-host agent, and cluster monitor — running agentic post-training and evaluation rollouts on the 3FS distributed filesystem. The training pipeline fed it a two-stage paradigm (domain specialist models trained with SFT+RL, then unified by on-policy distillation) with agent capability as an explicit domain, and the mid-training data mix gained agentic and long-document corpora. The pattern to notice: by 2026, DeepSeek treats agent-loop context behavior as a first-class training objective with its own data, infrastructure, and evaluation harness — not a post-deployment afterthought.

12.4 Quick Instruction: auxiliary tasks that reuse the cache

The V4 report's most quotable piece of systems craft targets a cost nobody benchmarks: the small model that production chat products run before the main model to classify the request (does this need a web search? what domain is it? should this URL be fetched?). That model cannot share the main model's KV cache, so the whole prefix is prefilled twice. Quick Instruction eliminates the second prefill by appending dedicated special tokens directly to the input sequence — <|action|>, <|title|>, <|query|>, <|authority|>, <|domain|>, <|read_url|> — each bound to one auxiliary task. Because the tokens ride on the already-computed KV cache, the auxiliary outputs (search-trigger decisions, generated queries, authority and domain classifications, URL-read decisions) are produced in the same forward pass, in parallel, "completely avoiding redundant prefilling," cutting user-perceived time-to-first-token, and retiring the extra model from the stack. It is the same insight as Manus's stable-prefix rule, inverted: instead of protecting the cache from auxiliary work, move the work into the cache. The same release also moved tool-call serialization to an XML format that "mitigates escaping failures and reduces tool-call errors" — protocol-level context hygiene, the unglamorous kind that production uptime is made of.

from openai import OpenAI

client = OpenAI(api_key="<DeepSeek API Key>",
                base_url="https://api.deepseek.com")

messages = [{"role": "user", "content": "Find and fix the failing test."}]

while True:
    resp = client.chat.completions.create(
        model="deepseek-flash",          # V4.1-Flash: $0.006/M cache hits (peak)
        messages=messages,
        tools=TOOLS,                       # tools present => agent semantics
        reasoning_effort="high",
    )
    msg = resp.choices[0].message

    # CRITICAL: keep reasoning_content in the history.
    # Dropping it => HTTP 400. Keeping it => reasoning persists
    # across tool outputs, per V3.2 Thinking Context Management.
    messages.append({
        "role": "assistant",
        "content": msg.content,
        "reasoning_content": msg.reasoning_content,
        "tool_calls": msg.tool_calls,
    })

    if not msg.tool_calls:
        break                            # final answer; loop ends

    for tc in msg.tool_calls:        # execute each call, cap the output
        result = execute(tc.function.name, json.loads(tc.function.arguments))
        messages.append({
            "role": "tool",
            "tool_call_id": tc.id,
            "content": cap(result, max_tokens=1500),  # P3: stable caps, -38% cost
        })
        # A new USER message here would trigger the discard rule;
        # tool outputs alone never do (§11.1, Rule 1).

def maybe_compact(messages, window, budget):
    used = tokens(messages)
    if used < 0.80 * window:          # V3.2 §4.4 trigger point
        return messages

    kept = [m for m in messages if m["role"] in ("system",)] \
        + [m for m in messages[-5:]]     # recent turns + system prompt

    summary = llm_summarize(
        messages,
        prompt="""Preserve: architectural decisions, unresolved bugs,
        implementation details, user constraints. Discard: raw tool outputs
        whose facts you have restated, superseded search results.
        For every dropped artifact, keep its reference (path/URL) so the
        agent can re-fetch it.""")                  # recall first, then precision

    return kept[:1] + [{"role": "user", "content": summary}] + kept[1:]

# Measured trade-offs before you ship this:
#  - DeepSeek BrowseComp study: Summary hit 60.2 at 364 avg steps;
#    Discard-all hit 67.6 with far fewer steps (§11.4).
#  - Towards AI: with cheap cache hits, keep-everything beat every
#    summary on cost+latency+recall at once (§3.2).
#  - Compaction rewrites the prefix: the next call re-prefills at full price.

{
  "persona": {
    "block": "You are the repo-maintenance agent for api-gateway.",
    "last_rewritten": "2026-09-13T10:22:31Z"
  },
  "task_state": {
    "block": "Goal: fix flaky auth e2e. Done: repro, log capture. Open: token-refresh race in middleware/auth.py:L214. Hypothesis 2 rejected (clock skew). Next: instrument refresh window.",
    "limit_tokens": 2000,              # bounded, in-window, always visible
    "rewrite_policy": "agent-tool: core_memory_replace"
  },
  "project_facts": {
    "block": "CI: GitHub Actions, runners have 7GB RAM. Test DB: ephemeral Postgres 16. Owner: @infra-oncall.",
    "limit_tokens": 1000
  }
}
# The todo.md recitation rule (Manus) is this pattern's file-shaped twin:
# rewrite the block every few steps so the plan sits at the END of context,
# where attention is strongest (§2.1) and drift dies.