How to Build an AI Agent

§ Abstract & How to Read This Guide

What this guide claims, where each step comes from, and how the evidence is organized.

Ask ten engineers how to build an AI agent and most will point at the same few documents. Anthropic's Building effective agents, published December 19, 2024 by Erik Schluntz and Barry Zhang, distilled lessons from working with dozens of teams across industries into one conclusion: "the most successful implementations weren't using complex frameworks or specialized libraries. Instead, they were building with simple, composable patterns." OpenAI's A practical guide to building agents, a 34-page distillation of customer deployments, lands on the same three-part anatomy — an agent is a model, plus tools, plus instructions — and the same warning about complexity. Hugging Face's free Agents Course teaches the same loop with smolagents, LlamaIndex, and LangGraph. The industry did not fragment into competing methodologies; it converged on one: start simple, wrap a capable model in a small set of well-designed tools, run it in a feedback loop, and add complexity only when measurement proves you need it.

This guide assembles that consensus into a single build. It is written in September 2026, when the agent stack has matured past its experimental phase — the Model Context Protocol Anthropic open-sourced in November 2024 was donated to the Linux Foundation's Agentic AI Foundation in December 2025, co-founded with Block and OpenAI; OpenAI's Agents SDK gained native sandboxing for long-horizon work in April 2026; Anthropic's Managed Agents made the hosted runtime a product that same month — and when the top of the SWE-bench Verified leaderboard is decided by single percentage points between harnesses built on the patterns in those two canonical guides. The steps are numbered and sequential: Step 1 decides whether you should build anything at all. Steps 2–7 build the system: model, tools, instructions, the loop, the context strategy, and the workflow patterns. Steps 8–10 make it trustworthy: guardrails, human oversight, evaluation. Steps 11–13 scale it: multi-agent orchestration, framework choice, and shipping.

How the guide is organized

Part I is everything you must settle before writing code: what an agent is (three definitions from the three canonical sources, and where they agree), whether your use case justifies one, and the architecture every production system shares — plus the 2022–2026 timeline that produced it. Part II is the build itself: one chapter per component, ending in ~80 lines of working Python that implement the entire loop. Part III covers the machinery that separates a demo from a system people trust: the seven guardrail types from OpenAI's guide, the two human-intervention triggers, and the evaluation harness that should exist before the first tool is written. Part IV handles scale: when a single agent stops being enough, the manager and decentralized orchestration patterns, the 2026 framework landscape with each tool's own positioning, and a final production checklist.

Three audiences were kept in mind. If you are building your first agent, read linearly — the steps are ordered so each one only depends on the ones before it. If you already run agents in production, Parts III and IV plus Chapter 9 (context engineering) are where the material your prototype lacks tends to live. If you are making a build-vs-buy decision, Chapter 3's scoring rubric and Chapter 15's framework table are written for exactly that meeting.

1 The Method Everyone References

Why the industry converged on a handful of documents, and what this guide takes from each.

Agent tutorials multiply daily, but the documents practitioners actually cite — in design docs, in conference talks, in the README of every serious agent framework — are few. The center of gravity is a single essay: Anthropic's "Building effective agents," published December 19, 2024 by Erik Schluntz and Barry Zhang. Its opening paragraph carries the most-quoted finding in the applied-agents literature: working with dozens of teams across industries, Anthropic found "the most successful implementations weren't using complex frameworks or specialized libraries. Instead, they were building with simple, composable patterns." That sentence reframed the field. In an ecosystem then dominated by elaborate multi-agent choreography frameworks, the lab behind Claude told developers to build less — and to start directly against model APIs.

The second pillar is OpenAI's "A practical guide to building agents," a 34-page guide distilled "from numerous customer deployments into practical and actionable best practices," with worked examples in OpenAI's Agents SDK. It formalizes what a first agent is made of — the guide's own summary: "In its most fundamental form, an agent consists of three core components: Model, the LLM powering the agent's reasoning and decision-making; Tools, external functions or APIs the agent can use to take action; Instructions, explicit guidelines and guardrails defining how the agent behaves." Where Anthropic's essay gives you the architectural patterns, OpenAI's guide gives you the assembly order and the operational safety net: model selection strategy, instruction-writing technique, orchestration patterns, and a full guardrail taxonomy.

The third is a teaching artifact rather than a vendor essay: Hugging Face's AI Agents Course, a free curriculum that walks from agent theory through hands-on building with smolagents, LlamaIndex, and LangGraph, with graded assignments and a community challenge. Its companion blog post, "Introducing smolagents" (December 31, 2024, by Aymeric Roucher, Merve Nabilouine, and Thomas Wolf), contributes the cleanest one-line definition in circulation — "AI Agents are programs where LLM outputs control the workflow" — and the agency spectrum that this guide uses in Chapter 2. Beneath all three sits the research paper that started the loop in the first place: ReAct (Yao et al., October 2022), whose 15,000-plus citations make it the most-cited ancestor of every agent shipped since.

What "the method" actually says

Stripped to its load-bearing rules, the consensus method is short enough to memorize. First, find the simplest solution possible, and only increase complexity when needed — Anthropic's phrasing, and the instruction the guide's Step 1 exists to enforce. Second, the unit of construction is the augmented LLM: a model plus retrieval, tools, and memory, wrapped behind a clean, well-documented interface. Third, the agent is a loop, not a graph: "typically just LLMs using tools based on environmental feedback in a loop," in Anthropic's words, running until an exit condition fires. Fourth, tools are a first-class engineering surface — Anthropic's teams spent more time optimizing tools than prompts on their SWE-bench agent, and they tell you to spend "just as much effort" on the agent-computer interface as human engineers spend on the human one. Fifth, trust is built in layers: guardrails, human intervention points, and an evaluation harness are not post-launch patches but parts of the original design.

2 What an Agent Actually Is

Three definitions from the three canonical sources, one spectrum, and the paper that started it.

Precision here is not pedantry — it is the difference between a project that ships and one that sprawls. The canonical sources define the term three ways, and the definitions agree more than they differ.

Definition one: the architectural split (Anthropic)

Anthropic categorizes everything as agentic systems, then draws one line that the rest of this guide inherits: "Workflows are systems where LLMs and tools are orchestrated through predefined code paths. Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks." The distinction is who decides the path. In a workflow, your code decided — before deployment — which LLM call happens next. In an agent, the model decides, turn by turn, from inside the run. This is why workflows are predictable and cheap, and agents are flexible and risky: you are trading a control flow you can read for one that is generated at runtime.

Definition two: independence on your behalf (OpenAI)

OpenAI's guide compresses it further: "Agents are systems that independently accomplish tasks on your behalf." The negative space matters just as much — "Applications that integrate LLMs but don't use them to control workflow execution—think simple chatbots, single-turn LLMs, or sentiment classifiers—are not agents." And the guide adds two core characteristics that every trustworthy agent possesses: it uses an LLM to manage workflow execution and make decisions, recognizing completion and proactively correcting its own actions, halting and transferring control back to the user on failure; and it has access to tools to interact with external systems, "always operating within clearly defined guardrails." Note the sequence in that last clause — guardrails are part of the definition, not an optional extra.

Definition three: the agency spectrum (Hugging Face)

smolagents dissolves the binary entirely: "AI Agents are programs where LLM outputs control the workflow. Any system leveraging LLMs will integrate the LLM outputs into code. The influence of the LLM's input on the code workflow is the level of agency of LLMs in the system." Agency, on this view, "evolves on a continuous spectrum, as you give more or less power to the LLM on your workflow." The published spectrum is the cleanest ladder in the literature for locating your own design:

The research root: ReAct

The loop all three sources assume traces to one paper. ReAct — "Synergizing Reasoning and Acting in Language Models," submitted to arXiv on October 6, 2022 and published at ICLR 2023 — proposed that an LLM generate "both reasoning traces and task-specific actions in an interleaved manner, allowing for greater synergy between the two: reasoning traces help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with external sources, such as knowledge bases or environments, to gather additional information." The results still shape the design intuition: on HotpotQA and Fever, interleaving search actions with reasoning "overcomes issues of hallucination and error propagation prevalent in chain-of-thought reasoning", and on the interactive benchmarks ALFWorld and WebShop, ReAct beat imitation- and reinforcement-learning baselines "by an absolute success rate of 34% and 10% respectively, while being prompted with only one or two in-context examples." The modern tool-calling API is, in effect, ReAct industrialized: the reasoning trace became the model's output tokens, and the action became a structured tool_use block instead of a parsed text string.

The synthesis: the augmented LLM

Anthropic's essay names the unit every pattern composes: "The basic building block of agentic systems is an LLM enhanced with augmentations such as retrieval, tools, and memory. Our current models can actively use these capabilities—generating their own search queries, selecting appropriate tools, and determining what information to retain." The essay's implementation advice is to "focus on two key aspects": tailor the augmentations to the use case, and "ensure they provide an easy, well-documented interface for your LLM" — for which it names the Model Context Protocol as one standard route. From here on, this guide assumes exactly that unit, and Part II is devoted to building it well.

3 Step 1 — Decide Whether You Should Build One

The gate every canonical source puts in front of the code: most use cases should not get an agent.

The first step of building an agent is refusing to build one, if you can help it. Anthropic's essay states the rule as a design axiom: "we recommend finding the simplest solution possible, and only increasing complexity when needed. This might mean not building agentic systems at all. Agentic systems often trade latency and cost for better task performance, and you should consider when this tradeoff makes sense." The corollary is easy to miss: "For many applications, however, optimizing single LLM calls with retrieval and in-context examples is usually enough." A RAG pipeline, a classifier, or a well-prompted single call is cheaper, faster, more predictable, and dramatically easier to debug than any loop. The agent exists for the cases where that stops being true.

When the trade starts paying: OpenAI's three criteria

OpenAI's guide devotes its own early chapter to use-case selection and gives the most operational test in the canon. Agents belong where "traditional deterministic and rule-based approaches fall short", and specifically where at least one of three frictions is present:

• Complex decision-making: workflows "involving nuanced judgment, exceptions, or context-sensitive decisions" — the guide's example is refund approval in customer service, where a rigid policy engine either rubber-stamps too much or escalates too much.
• Difficult-to-maintain rules: systems "that have become unwieldy due to extensive and intricate rulesets, making updates costly or error-prone" — its example is vendor security review, where the ruleset grows faster than anyone can audit it.
• Heavy reliance on unstructured data: scenarios that involve "interpreting natural language, extracting meaning from documents, or interacting with users conversationally" — its example is processing a home insurance claim from photos, free text, and adjuster notes.

The guide's archetype for the contrast is payment fraud analysis: a rules engine "works like a checklist, flagging transactions based on preset criteria," while an LLM agent "functions more like a seasoned investigator, evaluating context, considering subtle patterns, and identifying suspicious activity even when clear-cut rules aren't violated." The flip side is stated plainly: "Before committing to building an agent, validate that your use case can meet these criteria clearly. Otherwise, a deterministic solution may suffice." If you cannot name the specific decision your agent will make that no rule you can write would make, stop here.

The regularizer: bias against agency

Hugging Face's smolagents post adds the sharpest version of the discipline. Because an LLM in the control path injects nondeterminism into a workflow, the post advises engineers to "regularize towards not using any agentic behaviour" — a deliberately statistical phrasing: treat agency as a cost you add only under pressure of evidence. Its worked example is a surfing-trip website: if requests fall into two known buckets (browse trips, talk to sales), code the branches. "If that deterministic workflow fits all queries, by all means just code everything! This will give you a 100% reliable system with no risk of error introduced by letting unpredictable LLMs meddle in your workflow." The trigger to revisit is equally concrete: "If the pre-determined workflow falls short too often, that means you need more flexibility" — such as a traveler whose dates, passport delay, and cancellation terms interlock in ways no form anticipates. Flexibility is purchased when the rails demonstrably fail, not when the demo looks impressive.

Where agents demonstrably pay: the two proven domains

Anthropic's Appendix 1, "Agents in practice," names the two domains where customer evidence is strongest, and the shared anatomy is instructive. Customer support: conversations flow naturally while pulling external state and taking actions — tools retrieve order history and knowledge articles, refunds and ticket updates execute programmatically, and "success can be clearly measured through user-defined resolutions," to the point that several companies price the product on successful resolutions only. Coding agents: solutions are verifiable through automated tests, agents iterate on test results as feedback, the problem space is structured, and "output quality can be measured objectively." Both combine conversation and action, clear success criteria, real feedback loops, and meaningful human oversight — the four properties to look for in your own candidate use case. Klarna's support agent, cited by OpenAI at DevDay 2025, "handles two-thirds of all tickets"; that is what the pattern looks like at scale.

4 The Canonical Architecture

One loop, five components, and the four-year research arc that produced them.

The loop, precisely

Strip every framework away and the thing you are left building is small enough to hold in your head. smolagents publishes it as four lines of pseudocode, and it is the honest skeleton of every agent in production:

memory = [user_defined_task]
while llm_should_continue(memory):        # this loop is the multi-step part
    action = llm_get_next_action(memory)  # this is the tool-calling part
    observations = execute_action(action)
    memory += [action, observations]

OpenAI's guide describes the same construct in API terms: "Every orchestration approach needs the concept of a 'run', typically implemented as a loop that lets agents operate until an exit condition is reached." In the Agents SDK that loop is Runner.run(), which "loops over the LLM until either: a final-output tool is invoked, defined by a specific output type; [or] the model returns a response without any tool calls (e.g., a direct user message)." Anthropic adds the safety clauses that turn a loop into an engineered system: agents need "ground truth from the environment at each step (such as tool call results or code execution) to assess its progress," they can "pause for human feedback at checkpoints or when encountering blockers," and they should terminate "upon completion, but it's also common to include stopping conditions (such as a maximum number of iterations) to maintain control." Five components, then, and one boundary:

How the loop got here: 2022–2026

The architecture feels inevitable only in retrospect; it was assembled piece by piece over four years. Knowing which piece came from where tells you which parts of your stack are load-bearing research and which are conveniences that may be replaced.

One more property of this history is worth internalizing before Part II: nothing in the canonical architecture requires any specific vendor. The essays were written by model providers, but the method — augmented LLM, loop, exit conditions, guardrails, evaluation — is model-agnostic, and Chapter 5's model table deliberately spans four labs. Build the pattern, not the allegiance.

5 Step 2 — Choose the Model

The one decision that dominates your agent's ceiling — and the ladder OpenAI prescribes for making it.

Start at the top, then climb down

The model you pick is the single largest determinant of what your agent can do; every other component is scaffolding around it. OpenAI's guide gives a three-step procedure that has become standard practice, and its logic is anti-intuitive enough to be worth stating exactly: "An approach that works well is to build your agent prototype with the most capable model for every task to establish a performance baseline. From there, try swapping in smaller models to see if they still achieve acceptable results. This way, you don't prematurely limit the agent's abilities, and you can diagnose where smaller models succeed or fail." The summarized principles:

1. Set up evals to establish a performance baseline — before optimizing anything, know what "good" measures as.
2. Focus on meeting your accuracy target with the best model available.
3. Optimize for cost and latency by replacing larger models with smaller ones where possible.

The sequencing matters. Teams that start with a small model to "save money" cannot distinguish their harness's bugs from the model's limits; teams that prototype with the strongest model and then swap down know exactly which capability they are trading away at each price point. And the guide is explicit that "not every task requires the smartest model — a simple retrieval or intent classification task may be handled by a smaller, faster model, while harder tasks like deciding whether to approve a refund may benefit from a more capable model." In practice this extends inside a single agent: the model that orchestrates tool calls and plans needs to be stronger than the one that classifies a ticket or drafts a template, and routing between them is a standard pattern (Ch. 10).

What "best for agents" measured as, September 2026

Agentic capability now has public scoreboards, and the one with the longest track record is SWE-bench Verified — the human-filtered 500-instance subset of SWE-bench (originally 2,294 real GitHub issues, Jimenez et al., ICLR 2024), where an agent receives an issue description and must produce a patch that passes the repository's own tests. That makes it an end-to-end agent evaluation: reasoning, tool use, editing, and iteration are all measured through the outcome. The current leaderboard, bash-only setting, is a snapshot of the frontier:

The cleanest controlled comparison comes from holding the harness fixed. The leaderboard's mini-SWE-agent entries run one minimal agent loop for every model — which isolates the model as the variable, and exposes the cost spread at equal capability:

Capability checklist for an agent model

Benchmarks are a proxy; the properties that actually decide whether a model can drive your loop are more specific. The canonical sources converge on four. Reliable structured tool calling: the model must emit well-formed arguments natively and choose between similar tools accurately — the function-calling layer that arrived in June 2023 and has been trained-on ever since. Long-context discipline: agentic runs accumulate tens of thousands of tokens of tool results; degradation with filled windows ("context rot") hits agents harder than chat. Instruction adherence over long horizons: Anthropic's announcement for Claude Sonnet 4.5 (Sep 29, 2025) markets exactly this — then-best SWE-bench Verified (77.2%) and OSWorld computer-use (61.4%, up from 42.2% four months prior) scores, achieved in multi-hour, multi-turn runs. Error recovery: an agent that cannot read a tool error and change strategy will loop; test candidates by feeding them a failing tool result and watching the next action. When OpenAI's guide notes you can use advanced models "like o1 or o3-mini" to even generate your instructions from existing documents, it is the same principle at a different layer: use the strongest reasoning where reasoning is the bottleneck.

6 Step 3 — Design the Tools

The component the canonical sources agree you will spend the most time on — because it is where agents actually fail.

Anthropic's engineering essay on tooling opens with the sentence every agent builder should tape to the monitor: "Agents are only as effective as the tools we give them." And their SWE-bench team's confession is even more pointed — while building their coding agent, "we actually spent more time optimizing our tools than the overall prompt." This chapter assembles the tool-design doctrine from the three primary sources: OpenAI's guide (what tools to build), Anthropic's two essays (how to shape them), and the SWE-agent line of research (why the interface itself is the product).

The three tool types

OpenAI's guide classifies agent tools into three buckets, and the classification is useful precisely because it forces you to notice which permissions each bucket implies:

The guide's hygiene rules travel with the taxonomy: "Each tool should have a standardized definition, enabling flexible, many-to-many relationships between tools and agents," and well-documented, thoroughly tested, reusable tools "improve discoverability, simplify version management, and prevent redundant definitions." Where no API exists — legacy systems behind a GUI — the guide points to computer-use models that interact through the web or application UI "just as a human would," an escape hatch that trades structure for coverage. And it sets the escalation threshold you will meet again in Chapter 14: "As the number of required tools increases, consider splitting tasks across multiple agents."

Format rules: write for a token predictor

Anthropic's Appendix 2 ("Prompt engineering your tools") contains the most underrated insight in the canon: tool formats are not equivalent. The essay walks the case — you can specify a file edit as a diff or a full rewrite, return code in JSON or in markdown — and observes that although these are losslessly interconvertible in software terms, "some formats are much more difficult for an LLM to write than others." Writing a diff "requires knowing how many lines are changing in the chunk header before the new code is written"; code inside JSON "requires extra escaping of newlines and quotes." The three resulting rules, verbatim:

• "Give the model enough tokens to 'think' before it writes itself into a corner." Output-token limits and format terseness interact; a model squeezed into a terse format spends its reasoning budget on formatting instead of the task.
• "Keep the format close to what the model has seen naturally occurring in text on the internet." LLMs are next-token predictors trained on human text; formats far from natural distributions are out-of-distribution by construction.
• "Make sure there's no formatting 'overhead'" — no line counting, no string escaping of code. The interface should not tax the model's arithmetic for the privilege of acting.

The ACI doctrine

Both the essay and the follow-up research assign this a name and a budget. The rule of thumb: "think about how much effort goes into human-computer interfaces (HCI), and plan to invest just as much effort in creating good agent-computer interfaces (ACI)." The September 2025 engineering essay sharpens the mental model: "Tools are a new kind of software which reflects a contract between deterministic systems and non-deterministic agents." A weather function called by another program has a caller that reads the docstring once; the same function called by a model has a caller that re-reads the description on every inference and may respond to "Should I bring an umbrella?" by calling the tool, answering from memory, or asking a clarifying question. You are not writing an API for a deterministic consumer — you are writing for "a new hire on your team," in the essay's phrasing, which is why the essay recommends describing tools the way you would onboard that new hire: "Consider the context that you might implicitly bring — specialized query formats, definitions of niche terminology, relationships between underlying resources — and make it explicit."

The research lineage validates the doctrine with measurements. SWE-agent (Yang et al., NeurIPS 2024; 2,500+ citations) built its entire result on the observation that "SWE-agent's custom agent-computer interface (ACI) significantly enhances an agent's ability to create and edit code files, navigate entire repositories" — same model, purpose-built interface, dramatically better autonomous software engineering. And Anthropic reports that "Claude Sonnet 3.5 achieved state-of-the-art performance on the SWE-bench Verified evaluation after we made precise refinements to tool descriptions, dramatically reducing error rates and improving task completion." Tool descriptions are not documentation; they are the highest-leverage prompt surface in the system.

Shape the returns: tokens are the currency

The September 2025 essay devotes a full section to what tools send back. The headline number: "For Claude Code, we restrict tool responses to 25,000 tokens by default." Even as context windows grow, the essay predicts "the need for context-efficient tools to remain" — because every returned token competes for the attention budget Chapter 9 shows is scarce. The recommended mechanics: "implementing some combination of pagination, range selection, filtering, and/or truncation with sensible default parameter values for any tool responses that could use up lots of context," and steering agents toward "many small and targeted searches instead of a single, broad search." The essay's worked example offers two response modes for a Slack tool — "detailed" (includes IDs needed for follow-up calls) and "concise" (content only) — where the concise variant uses roughly one-third the tokens. Response structure itself (XML, JSON, or Markdown) "can have an impact on evaluation performance: there is no one-size-fits-all solution" — so test it against your eval rather than following fashion.

Error returns deserve the same care. The contrast the essay draws is between an opaque traceback and a response that tells the agent what to change — input-validation failures should "clearly communicate specific and actionable improvements." An error message is a steering signal for the next loop iteration; treat writing it as prompt engineering, not logging. On naming: "input parameters should be unambiguously named: instead of a parameter named user, try a parameter named user_id." And on boundaries, namespacing: give related tools a shared prefix so the model can cluster "which tools belong to which functionality" — the essay identifies clear namespacing as a key principle for keeping tool sets legible as they grow.

tools = [{
    "name": "search_orders",
    "description": (
        "Full-text search over customer orders. Returns the order_id, status, "
        "total, and updated_at for each match - plus the next_page token when more "
        "results exist. Use narrow queries (email or order number) rather than broad "
        "ones; results are capped at 10 per page."          # explicit, steers strategy
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "query": {"type": "string",
                "description": "Search term: an order number or customer email"},
            "status_filter": {"type": "string", "enum": ["open", "shipped", "refunded"]},
            "page_token": {"type": "string"}
        },
        "required": ["query"]
    }
}]

def search_orders(query, status_filter=None, page_token=None):
    if len(query) < 3:
        return {"error": "query too short - include the customer email "
                     "or the order number you are looking for"}  # actionable error
    rows, next_token = order_db.search(query, status_filter, page_token, limit=10)
    return {"results": [compact(r) for r in rows],   # trimmed fields only
            "next_page": next_token,                   # pagination, not dumps
            "hint": "pass next_page to continue"}

Buy the plumbing: MCP

Since November 25, 2024, the integration layer has a standard. Anthropic's Model Context Protocol announcement described the problem it solves — "every new data source required its own custom implementation, making truly connected systems difficult to scale" — and the fix: "a universal, open standard for connecting AI systems with data sources, replacing fragmented integrations with a single protocol." The architecture splits cleanly: you expose data through MCP servers and consume them from MCP clients inside your agent, with pre-built servers shipping from day one for Google Drive, Slack, GitHub, Git, Postgres, and Puppeteer. Early adopters included Block and Apollo, with development-tool makers Zed, Replit, Codeium, and Sourcegraph integrating to "enable AI agents to better retrieve relevant information" for coding tasks. The protocol's trajectory since is the strongest signal of industry consensus: on December 9, 2025, Anthropic donated MCP to the Linux Foundation's newly formed Agentic AI Foundation, co-founded with Block and OpenAI (which contributed its own AGENTS.md convention) — moving the standard to neutral governance. For you, at Step 3, the practical meaning is simple: before writing a tool, check whether an MCP server already exists for the system you need; your tool layer can be configuration, not code.

A closing meta-technique from the tools essay completes the doctrine: use agents to build agent tools. Anthropic's team iterates tool quality by having Claude Code itself run the evaluation and propose improvements — "you can use Claude Code to automatically optimize its tools for itself" — and reports finding issues from "contradictory tool descriptions to inefficient tool implementations and confusing tool schemas" this way. The tool layer is engineering, but it is also the most automatable layer of the whole stack.

7 Step 4 — Write the Instructions

The system prompt as policy document: OpenAI's four practices, and the routine format they converge on.

Instructions are the third component of OpenAI's anatomy, and the guide is blunt about their weight: "High-quality instructions are essential for any LLM-powered app, but especially critical for agents. Clear instructions reduce ambiguity and improve agent decision-making, resulting in smoother workflow execution and fewer errors." In a chat product a vague sentence costs one reply; in an agent loop it is multiplied by every turn and compounded by tool consequences. The guide's four practices for agent instructions:

• Use existing documents. "When creating routines, use existing operating procedures, support scripts, or policy documents to create LLM-friendly routines" — in customer service, "routines can roughly map to individual articles in your knowledge base." Your organization already wrote most of the policy; the job is translation, not invention.
• Prompt agents to break down tasks. "Providing smaller, clearer steps from denser resources helps minimize ambiguity and helps the model better follow instructions" — decomposed steps also give your evaluation (Ch. 13) observable intermediate states.
• Define clear actions. "Make sure every step in your routine corresponds to a specific action or output" — down to the wording of user-facing messages. "Being explicit about the action leaves less room for errors in interpretation."
• Capture edge cases. "Real-world interactions often create decision points" — incomplete information, unexpected questions, alternative branches when a required piece of information is missing. "A robust routine anticipates common variations."

Two force-multipliers from the guide complete the practice. First, you can generate a first draft mechanically: "You can use advanced models, like o1 or o3-mini, to automatically generate instructions from existing documents" — the guide supplies the prompt, which asks the model to "convert the following help center document into a clear set of instructions, written in a numbered list... Ensure that there is no ambiguity." Second, scale one prompt across use cases with prompt templates: rather than maintaining many per-case prompts, "use a single flexible base prompt that accepts policy variables" — the guide's call-center example parameterizes the user's first name, tenure, and complaint categories into one template, so new use cases become variable updates rather than prompt rewrites.

SYSTEM_PROMPT = """
You are a customer support agent for a software company.

# Routine
1. Identify what the customer needs. If the request is ambiguous, ask one
   clarifying question - do not guess.
2. Look up the account with search_orders (customer email or order number)
   before making any statement about order status.
3. For refunds: confirm the order and amount with the customer first, then
   call issue_refund. Never issue refunds over $200 - escalate instead.
4. Reply in at most five sentences. State what you did, and what happens next.

# Edge cases
- Customer provides no order reference: ask for the email used at checkout.
- Order not found: say so, and offer to search by email instead.
- Customer is angry: acknowledge once, then proceed with the routine.
"""

Anthropic adds a constraint the OpenAI guide leaves implicit: the system prompt is also a context budget item. Their agent principles include "prioritize transparency by explicitly showing the agent's planning steps" — ask the model to state its next action and why, which costs tokens but buys debuggability and catches runaway plans at the human checkpoint (Ch. 12). And both guides converge on the same meta-rule this whole chapter instantiates: write instructions the way you wrote tools — for a capable but literal newcomer with no memory of your organization's folklore.

8 Step 5 — Build the Loop

Everything so far, assembled into ~80 lines of working Python — the part frameworks would have you believe is hard.

This is the step where the mythology dies. The loop — the thing entire frameworks wrap in abstraction — is a while-loop over a message array, and writing it yourself once is the single best way to understand every system this guide will later ask you to choose between. The implementation below uses Anthropic's Messages API, the documented tool-use pattern; every other vendor's SDK has an isomorphic shape. The schemas and instruction prompt come from Chapters 6 and 7 unchanged:

import json, anthropic

client = anthropic.Anthropic()

MAX_TURNS = 25                     # stopping condition: Anthropic's "maximum iterations"
TOOL_IMPLS = {"search_orders": search_orders, "issue_refund": issue_refund}

def run_agent(user_message: str) -> str:
    messages = [{"role": "user", "content": user_message}]
    for turn in range(MAX_TURNS):
        response = client.messages.create(
            model="claude-sonnet-4-5",      # Ch.5: strongest model first, swap down later
            max_tokens=4096,
            system=SYSTEM_PROMPT,           # Ch.7: the routine + edge cases
            tools=TOOLS,                    # Ch.6: schemas are prompt surface
            messages=messages,
        )
        # EXIT 1: an answer with no tool call - the run is complete
        if response.stop_reason != "tool_use":
            return "".join(b.text for b in response.content if b.type == "text")

        # The model requested tools: record its turn, then execute.
        messages.append({"role": "assistant", "content": response.content})
        results = [execute_tool(b) for b in response.content if b.type == "tool_use"]
        messages.append({"role": "user", "content": results})   # ground truth re-enters the loop

    # EXIT 2: bounded work - never ship an agent without this
    return "Stopped: maximum turns reached without resolution."

def execute_tool(block) -> dict:
    """Run one tool call; errors become steering, not crashes (Ch. 6)."""
    try:
        impl = TOOL_IMPLS.get(block.name)
        if impl is None:
            content = (f"Unknown tool '{block.name}'. "
                       f"Available: {', '.join(TOOL_IMPLS)}.")
        else:
            content = json.dumps(impl(**block.input))
        return {"type": "tool_result", "tool_use_id": block.id,
                "content": content}
    except Exception as exc:
        return {"type": "tool_result", "tool_use_id": block.id,
            "is_error": True,
            "content": (f"Tool '{block.name}' failed: {exc}. "
                       "Fix the arguments and retry, or take a different approach.")}

Trace the loop once against the pseudocode from Chapter 4 and the canonical definitions line up exactly. llm_get_next_action is the API call; execute_action is execute_tool; memory += [action, observations] is the two append calls; llm_should_continue is stop_reason != "tool_use". The exit conditions are the two OpenAI enumerates for Runner.run() — "a final-output tool is invoked" or "the model returns a response without any tool calls" — plus the one Anthropic insists on: "stopping conditions (such as a maximum number of iterations) to maintain control." A production loop adds a third exit, error-abort, when a failure repeats beyond a retry budget — which is also the signal to hand off to a human (Ch. 12).

Three details that separate this from a toy

Assistant turns are appended before results. The full response.content — including any text the model wrote alongside its tool calls — goes into the message array first, then the tool results follow as the next user message. Dropping the assistant turn is the single most common hand-rolled bug: the model loses its own reasoning and repeats itself.
Errors are content, not exceptions. execute_tool converts failures into tool_result blocks with actionable text. The next inference sees the failure, reads the steering, and can change strategy — this is Anthropic's "recovering from errors" agent capability, implemented in one branch.
The environment is the teacher. No step of the loop plans from imagination: "it's crucial for the agents to gain 'ground truth' from the environment at each step (such as tool call results or code execution) to assess its progress." Every capability your agent appears to have is really the loop feeding observed consequences back — which is why tool quality (Ch. 6) dominates prompt quality, and why sandboxed testing environments matter before anything touches production (Ch. 11).

9 Step 6 — Engineer the Context

The step the canonical guides underweighted in 2024 and the field has since promoted to a discipline of its own.

Between publishing the agent guide (December 2024) and September 2025, Anthropic's engineering team watched long-horizon agents fail in a way tool design could not fix — and wrote the essay that gave the failure a name. "Effective context engineering for AI agents" (Sep 29, 2025) opens with the resource constraint every agent inherits: "Context is a critical but finite resource for AI agents." The discipline it defines: "Context engineering is the art and science of curating what will go into the limited context window from that constantly evolving universe of possible information." For your build, this step is the answer to a question the loop of Chapter 8 raises automatically: what exactly accumulates in messages after twenty turns, and what happens when the model has to reason over all of it?

Rot first: why more context is not more capability

The empirical grounding is uncomfortable. Across models, performance degrades as the window fills — the essay's phrasing: "While some models exhibit more gentle degradation than others, this characteristic emerges across all models. Context, therefore, must be treated as a finite resource with diminishing marginal returns. Like humans, who have limited working memory capacity, LLMs have an 'attention budget' that they draw on when parsing large volumes of context. Every new token introduced depletes this budget by some amount." Independent measurement agrees: Chroma's "Context Rot" study (July 2025) tested 18 models and found performance cliffs on reasoning-over-corporus tasks as relevant information density fell — positional drops exceeding 30 points in the worst regimes. An agent loop is a context-rot machine by construction: every turn appends planning text, tool schemas echo, raw tool results land wholesale. The problem compounds quietly — the agent still answers, just worse, which is why Chapter 13's evaluation must include long-horizon trajectories, not just single turns.

Technique one: compaction

For tasks that outrun the window, Anthropic's first lever: "Compaction is the practice of taking a conversation nearing the context window limit, summarizing its contents, and reinitiating a new context window with the summary. Compaction typically serves as the first lever in context engineering to drive better long-term coherence." The mechanics are simple — detect a threshold (say, 70% of the window), run a summarization call over the transcript that preserves goal state, decisions made, and open threads, then swap the transcript for the summary and continue. The design work is in what the summary must retain: task objective, verified facts from tool results, constraints discovered, and the current plan. Compaction loses detail by design; what you keep defines the agent's long-term coherence.

Technique two: structured note-taking

The essay's second technique moves durable state outside the window entirely: "structured note-taking," where "the agent regularly writes notes persisted to memory outside of the context window. These notes get pulled back into the context window at later times. This strategy provides persistent memory with minimal overhead." The canonical example is Claude Code's to-do list, or "your custom agent maintaining a NOTES.md file" — "this simple pattern allows the agent to track progress across complex tasks, maintaining critical context and dependencies that would otherwise be lost across dozens of tool calls." The demonstration that made it famous: Claude playing Pokémon, which "maintains precise tallies across thousands of game steps — tracking objectives like 'for the last 1,234 steps I've been training my Pokémon in Route 1, Pikachu has gained 8 levels toward the target of 10'" — unprompted, developing maps, unlocked achievements, and combat strategies in its own notes. In your build, this is one tool (write_note / read_notes) and one line in the instructions — the cheapest capability-per-line-of-code in the entire stack.

Technique three: sub-agents with small returns

The third technique is architectural: "Rather than one agent attempting to maintain state across an entire project, specialized sub-agents can handle focused tasks with clean context windows. The main agent coordinates with a high-level plan while subagents perform deep technical work or use tools to find relevant information. Each subagent might explore extensively, using tens of thousands of tokens or more, but returns only a condensed, distilled summary of its work (often 1,000-2,000 tokens)." The pattern is context isolation by structure: exploration cost is quarantined where it cannot rot the coordinator's window. It also composes with Chapter 14's multi-agent orchestration — a sub-agent is the manager pattern deployed as a memory strategy, one tool call wide.

Technique four: just-in-time retrieval

The fourth inverts the reflex of pre-loading knowledge. Instead of stuffing documents into the first turn, the agent fetches what it needs when it needs it. Claude Code is the reference hybrid: "CLAUDE.md files are naively dropped into context up front, while primitives like glob and grep allow it to navigate its environment and retrieve files just-in-time, effectively bypassing the issues of stale indexing and complex syntax trees." The trade is latency and engineering opinion: "runtime exploration is slower than retrieving pre-computed data," and the tool surface must be designed so exploration converges — the essay notes agents "assemble understanding layer by layer, maintaining only what's necessary in working memory." For read-heavy agents, a hybrid works best: a stable, small preamble (policies, schemas) up front; dynamic material fetched on demand.

The economics that make this a step, not a nicety

Two production numbers close the argument. Manus's engineering blog (July 2025) disclosed that its agent loops run at a 100:1 input-to-output token ratio — roughly 50 tool calls per task, each re-sending accumulated context — which is why the company treats KV-cache hit rate as a first-class engineering metric and instructs agents to mask stale tool outputs rather than delete them (deletion changes the prefix and invalidates the cache). And the 25,000-token tool-response cap from Chapter 6 is the same doctrine at the component level: cap what enters, not what the model must then carry. Every token you keep is paid for twice — once in inference cost on every subsequent turn, once in attention dilution. The essay's summary principle is the one to build against: "find the smallest set of high-signal tokens that maximize the likelihood of your desired outcome."

10 Step 7 — Compose the Workflows

Anthropic's five patterns, verbatim rules for when each one pays, and the graduation rule from workflow to agent.

Before granting an agent autonomy over an open-ended task, the canonical method routes the same components through predefined code paths — Anthropic's workflows. The essay is explicit that these are not rungs on a complexity ladder but a palette: "These building blocks aren't prescriptive. They're common patterns that developers can shape and combine to fit different use cases." Five patterns, each with its own when-to-use test:

Pattern 1 — Prompt chaining

"Prompt chaining decomposes a task into a sequence of steps, where each LLM call processes the output of the previous one. You can add programmatic checks (see 'gate' in the diagram below) on any intermediate steps to ensure that the process is still on track." Use when: "the task can be easily and cleanly decomposed into fixed subtasks. The main goal is to trade off latency for higher accuracy, by making each LLM call an easier task." The essay's examples: "generating marketing copy, then translating it into a different language" and "writing an outline of a document, checking that the outline meets certain criteria, then writing the document based on the outline." The gate is the pattern's quiet superpower — deterministic validation between stochastic steps, which is also where your guardrails (Ch. 11) naturally attach:

outline = call_llm("Write an outline for: " + brief)
if not outline_is_complete(outline):        # programmatic check - no LLM
    outline = call_llm("Fix the missing sections in: " + outline)
document = call_llm("Write the document from this outline: " + outline)

Pattern 2 — Routing

"Routing classifies an input and directs it to a specialized followup task. This workflow allows for separation of concerns, and building more specialized prompts. Without this workflow, optimizing for one kind of input can hurt performance on other inputs." Use when "complex tasks where there are distinct categories that are better handled separately, and where classification can be handled accurately." The essay's examples: directing "general questions, refund requests, technical support" into different downstream processes, prompts, and tools — and, notably, cost routing: "routing easy/common questions to smaller, cost-efficient models like Claude Haiku 4.5 and hard/unusual questions to more capable models like Claude Sonnet 4.5 to optimize for best performance." The classifier itself can be a cheap model or a traditional one; routing is where Step 2's model ladder becomes architecture.

Pattern 3 — Parallelization

Two variants: sectioning ("breaking a task into independent subtasks run in parallel") and voting ("running the same task multiple times to get diverse outputs"). Use when "the divided subtasks can be parallelized for speed, or when multiple perspectives or attempts are needed for higher confidence results" — and for the subtle reason the essay highlights: "LLMs generally perform better when each consideration is handled by a separate LLM call, allowing focused attention on each specific aspect." Examples: guardrails where "one model instance processes user queries while another screens them for inappropriate content" (which "tends to perform better than having the same LLM call handle both"), evals where "each LLM call evaluates a different aspect," and voting for "reviewing a piece of code for vulnerabilities, where several different prompts review and flag the code if they find a problem."

Pattern 4 — Orchestrator-workers

"A central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results." Use when "you can't predict the subtasks needed" — the essay's example: coding, "where the number of files that need to be changed and the nature of the change in each file likely depend on the task." The differentiator from parallelization: "subtasks aren't pre-defined, but determined by the orchestrator based on the specific input." Examples: "coding products that make complex changes to multiple files" and "search tasks that involve gathering and analyzing information from multiple sources." This is the pattern that most resembles an agent while remaining a workflow — the orchestrator plans, but the plan's topology is still bounded by your code, and workers run with clean context (Ch. 9's quarantine for free).

Pattern 5 — Evaluator-optimizer

"One LLM call generates a response while another provides evaluation and feedback in a loop." Use when "we have clear evaluation criteria, and when iterative refinement provides measurable value," with two signs of good fit: "LLM responses can be demonstrably improved when a human articulates their feedback," and "the LLM can provide such feedback" — "analogous to the iterative writing process a human writer might go through." Examples: "literary translation where there are nuances that the translator LLM might not capture initially," and "complex search tasks that require multiple rounds of searching and analysis, where the evaluator decides whether further searches are warranted."

The graduation rule

When does a workflow stop being enough? The essay's boundary: agents "can be used for open-ended problems where it's difficult or impossible to predict the required number of steps, and where you can't hardcode a fixed path," with the essential precondition that "you must have some level of trust in its decision-making." In practice you graduate when the branches multiply faster than you can maintain them — OpenAI's guide gives the two measurable triggers in its multi-agent chapter, and they apply a fortiori to the agent decision: prompts "contain many conditional statements (multiple if-then-else branches)" and tools have grown past what better naming cannot fix. Between workflow and agent sits the economics of error: a workflow fails locally (one step reruns); an agent compounds ("higher costs, and the potential for compounding errors" — hence "extensive testing in sandboxed environments"). The closing rule of the essay's pattern section is the one to obey: "you should consider adding complexity only when it demonstrably improves outcomes." If your eval cannot demonstrate it, the workflow stays.

11 Step 8 — Build the Guardrails

Seven guardrail types, one layering principle, and the execution model that keeps them from adding latency.

Guardrails entered the definition of an agent back in Chapter 2 — OpenAI's second core characteristic has agents "always operating within clearly defined guardrails" — and this step makes good on it. The guide's framing is risk-first: "Well-designed guardrails help you manage data privacy risks (for example, preventing system prompt leaks) or reputational risks (for example, enforcing brand-aligned model behavior)." Two boundaries matter before the taxonomy. Guardrails are a layer on top of ordinary security, not a substitute: "they should be coupled with robust authentication and authorization protocols, strict access controls, and standard software security measures." And no single check suffices: "Think of guardrails as a layered defense mechanism. While a single one is unlikely to provide sufficient protection, using multiple, specialized guardrails together creates more resilient agents."

The seven types

Note the architecture hiding in the table: three of the seven types are model-based (relevance, safety, output validation), two are deterministic (rules-based, PII), and the rest are policy glue. This is the parallelization pattern from Chapter 10 applied to safety — and the guide's execution model makes the link explicit. In the Agents SDK, "guardrails are first-class concepts, relying on optimistic execution by default. Under this approach, the primary agent proactively generates outputs while guardrails run concurrently, triggering exceptions if constraints are breached." Optimistic execution is why guardrails do not double your latency: the main path runs ahead, checks run alongside, and a tripwire aborts everything if a constraint fires. The sequence discipline for adopting them is three steps, verbatim: "Focus on data privacy and content safety" first; "add new guardrails based on real-world edge cases and failures you encounter"; then "optimize for both security and user experience, tweaking your guardrails as your agent evolves."

Tool safeguards: the risk map

The guardrail that most changes agent architecture is the one OpenAI attaches to tools: "Assess the risk of each tool available to your agent by assigning a rating—low, medium, or high—based on factors like read-only vs. write access, reversibility, required account permissions, and financial impact. Use these risk ratings to trigger automated actions, such as pausing for guardrail checks before executing high-risk functions or escalating to a human if needed." Implemented against Chapter 6's tool taxonomy, the mapping is mechanical: Data tools are read-only and rate low; Action tools with side effects rate by reversibility and blast radius (an email send is medium; issue_refund is high); Orchestration tools inherit the rating of what they wrap. The map lives in your execute_tool function from Chapter 8 — one lookup before dispatch, one branch to the approval queue.

Sandboxing: guardrails for the environment itself

When tools execute real commands and file writes, the strongest guardrail is not a classifier but a boundary. Anthropic's essay prescribes "extensive testing in sandboxed environments, along with the appropriate guardrails" before granting autonomy. The 2026 stack has productized this: OpenAI's April 15, 2026 Agents SDK evolution added a model-native harness and native sandbox execution added "a model-native harness and native sandbox execution" — "the sandbox provides compute: files, commands, packages, artifacts" — precisely so agents canmdash; "the sandbox provides compute: files, commands, packages, artifacts" added "a model-native harness and native sandbox execution" — "the sandbox provides compute: files, commands, packages, artifacts" — precisely so agents canmdash; precisely so agents can "inspect files, run commands, edit code, and work on long-horizon tasks" without touching your infrastructure; Anthropic's Agent SDK ships a permission system that "controls which tools run automatically, [and] which need approval" (Ch. 12); and its Managed Agents product runs the whole loop in hosted isolation. The rule of thumb: classify by blast radius. Read-only over public data → loop freely. Writes to your systems → sandbox or approval. Writes to the world → human approval, always, until Chapter 13's evals give you the confidence to relax it.

12 Step 9 — Put the Human in the Loop

The two escalation triggers every guide agrees on, and the checkpoint design that keeps humans effective, not bored.

Human oversight is not a temporary scaffold to be removed — both canonical guides treat it as a permanent architectural component. Anthropic's agents "pause for human feedback at checkpoints or when encountering blockers," beginning with the human command or discussion that starts the task itself. OpenAI devotes the closing section of its guide to it: "Human intervention is a critical safeguard enabling you to improve an agent's real-world performance without compromising user experience. It's especially important early in deployment, helping identify failures, uncover edge cases, and establish a robust evaluation cycle." The mechanism: "Implementing a human intervention mechanism allows the agent to gracefully transfer control when it can't complete a task" — for customer service, "escalating the issue to a human agent"; for a coding agent, "handing control back to the user."

The two triggers

OpenAI names exactly two conditions that warrant human intervention, and together they cover both failure modes and danger modes:

• Exceeding failure thresholds. "Set limits on agent retries or actions. If the agent exceeds these limits (e.g., fails to understand customer intent after multiple attempts), escalate to human intervention." This is the Chapter 8 max-turns exit wired to a person instead of an apology.
• High-risk actions. "Actions that are sensitive, irreversible, or have high stakes should trigger human oversight until confidence in the agent's reliability grows. Examples include canceling user orders, authorizing large refunds, or making payments."

The two triggers compose with the tool risk map from Chapter 11 into a single decision surface: every tool call carries a rating; ratings gate execution; failures exhaust budgets and route to a human. In Anthropic's Agent SDK this surface is a product feature — permissions "control which tools run automatically, [and] which need approval" — and the CLI exposes the same control interactively, which is why coding agents built on it visibly pause before destructive operations. The pattern to implement in your own loop is an interrupt: before dispatching a high-rated tool, park the run's state (messages, pending call), surface the request with context, and resume or abort on the human's answer. Resumability is what keeps humans optional in latency terms — the run waits, the rest of the system does not block.

One design rule closes the step, drawn from both guides' shared instinct: escalate early in deployment, relax on evidence. OpenAI's phrasing is "until confidence in the agent's reliability grows"; the evidence that grows confidence is exactly what the next step builds. Human-in-the-loop is not the admission that agents are weak — it is the measurement apparatus that makes them trustworthy enough to become autonomous, one eval at a time.

13 Step 10 — Evaluate Everything

The harness that should have existed since Step 2 — built now, before scale multiplies every defect you haven't measured.

Every previous step has deferred to this one. OpenAI's model-selection principle 1 ("set up evals to establish a performance baseline") puts evaluation before optimization; Anthropic's pattern section closes with "the key to success, as with any LLM features, is measuring performance and iterating on implementations"; the tools essay builds its whole optimization loop on top of an evaluation ("building an evaluation allows you to systematically measure the performance of your tools"). Evaluation is not the final step of building an agent — it is the step that makes every other step safe to repeat. A change to prompt, tool, model, or guardrail without an eval suite is a change you cannot distinguish from a regression.

What to measure: three layers

Agent evaluation has converged on a three-layer stack, and the layering matters because each layer catches failures the others cannot:

• Outcome metrics — did the task succeed? Task-completion rate, resolution quality, and cost per resolved task. These are the numbers your stakeholders see; they are also the slowest to debug from, because a failed outcome does not tell you where the run went wrong.
• Trajectory metrics — the complete ordered trace of what the agent did: plans, model decisions, tool calls and their arguments, results, turns taken, and recovery behavior. Trajectory evaluation "inspect[s] the complete ordered trace," and it is the only layer that can see the failure modes unique to agents: loops, redundant calls, tool misuse with correct outcomes, and degraded behavior as context fills (Ch. 9).
• Tool metrics — per-tool performance inside the run. Anthropic's tools essay lists the set: "the total runtime of individual tool calls and tasks, the total number of tool calls, the total token consumption, and tool errors," noting that "tracking tool calls can help reveal common workflows that agents pursue and offer some opportunities for tools to consolidate."

How to judge: LLM-as-judge and ground truth

Outcomes are easy to score when ground truth is objective — a test suite passes, an order exists, a refund posts. The canonical example is SWE-bench Verified itself: the 500-instance, human-filtered subset where an agent's patch is scored by whether it passes the repository's own tests, which is why it became the field's reference scoreboard (Ch. 5). For tasks where correctness is contextual — tone, completeness, helpfulness, "did the agent follow policy" — the working method is an LLM-as-judge: a separate model call, given the trace and a rubric, producing structured verdicts. The 2026 evaluation literature treats the two as complements: judges for subjective or context-dependent criteria, programmatic checks wherever an assertion can be written. Two rules make judges trustworthy: score with rubrics and examples rather than vibes (the same instruction discipline as Ch. 7), and spot-check a sample of verdicts by hand — a judge is a component that itself needs evaluation. Run offline on a fixed task set for regression testing, and online (sampling live traffic) for drift; the trace you log per turn (Ch. 8's production note) is the raw material for both.

When to run it: the regression gate

An eval suite only protects you if it runs on every change. The gate discipline: any edit to the system prompt, any tool schema change, any model swap (Ch. 5's ladder), any guardrail retuning (Ch. 11) triggers the full offline suite before deploy. This is also where agent-assisted optimization closes the loop, per the tools essay: once an evaluation exists, "you can use Claude Code to automatically optimize its tools against this evaluation" — having the agent iterate tool descriptions and implementations against the metric, with humans reviewing the diffs. Anthropic attributes measurable results to exactly this workflow: the SWE-bench improvements from "precise refinements to tool descriptions" were validated by evaluation, not intuition. The same loop generalizes: your eval suite plus an agent with edit access to prompts and schemas is a continuous optimization pipeline with a regression net underneath.

14 Step 11 — Orchestrate Multiple Agents

The rule is restraint, the two shapes are manager and handoff, and the trigger is measurable — not ambition.

Max out one agent first

OpenAI's guide states the rule before anything else: "Our general recommendation is to maximize a single agent's capabilities first. More agents can provide intuitive separation of concepts, but can introduce additional complexity and overhead, so often a single agent with tools is sufficient." A single agent is easier to evaluate (one trace, one scorecard), easier to debug (one prompt, one tool set), and cheaper (no inter-agent context duplication). The guide even gives the intermediate move before splitting: "use prompt templates" — one flexible base prompt with policy variables (Ch. 7) rather than N agents per use case. Splitting is justified by exactly two measurable signals:

• Complex logic. "When prompts contain many conditional statements (multiple if-then-else branches), and prompt templates get difficult to scale, consider dividing each logical segment across separate agents."
• Tool overload. "The issue isn't solely the number of tools, but their similarity or overlap. Some implementations successfully manage more than 15 well-defined, distinct tools while others struggle with fewer than 10 overlapping tools." The fix ladder: first try "providing descriptive names, clear parameters, and detailed descriptions" (Ch. 6's ACI work); split only "if improving tool clarity... doesn't improve performance."

Read those numbers again, because they invert the instinct: the ceiling is not ten tools — well-designed toolsets run past fifteen. If your agent is failing with eight, the diagnosis is tool design, not tool count. Only when clarity work plateaus does Chapter 14 actually begin.

Shape one: the manager pattern (agents as tools)

"A central 'manager' agent coordinates multiple specialized agents via tool calls, each handling a specific task or domain." The mechanics are exactly Chapter 6's orchestration tool type: specialists are wrapped as_tool(), so the manager "intelligently delegates tasks to the right agent at the right time, effortlessly synthesizing the results into a cohesive interaction" — and, critically, "instead of losing context or control," the manager keeps the conversation thread with the user. The guide's worked example is the translation trio (Spanish, French, Italian agents as tools of a manager agent), and the fit test is explicit: use the manager pattern for "workflows where you only want one agent to control workflow execution and have access to the user." One voice to the user, many hands behind it:

manager_agent = Agent(
    name="manager_agent",
    instructions="You are a translation agent. You use the tools given to you to translate.",
    tools=[
        spanish_agent.as_tool(tool_name="translate_to_spanish",
                              tool_description="Translate the user's message to Spanish"),
        french_agent.as_tool(tool_name="translate_to_french", ...),
        italian_agent.as_tool(tool_name="translate_to_italian", ...),
    ],
)
output = await Runner.run(manager_agent, "Translate 'hello' to Spanish, French and Italian for me!")

Shape two: decentralized handoffs

"Multiple agents operate as peers, handing off tasks to one another based on their specializations." A handoff is a one-way transfer: "If an agent calls a handoff function, we immediately start execution on that new agent that was handed off to while also transferring the latest conversation state." The guide's customer-service example is the canonical topology — a triage agent that assesses the query and hands off to a technical support, sales assistant, or order management agent, each with its own tools and instructions; control can optionally hand back. Use it "whenever you prefer specialized agents to fully take over certain tasks without the original agent needing to remain involved" — conversation triage, domain takeovers, shift changes. The graph view unifies the two shapes: "Multi-agent systems can be modeled as graphs, with agents represented as nodes. In the manager pattern, edges represent tool calls whereas in the decentralized pattern, edges represent handoffs that transfer execution between agents."

Choose the loop style: code-first or graph

The guide ends orchestration with a framework decision that has become a genuine fork in the 2026 ecosystem: "Some frameworks are declarative, requiring developers to explicitly define every branch, loop, and conditional in the workflow upfront through graphs consisting of nodes (agents) and edges (deterministic or dynamic handoffs). While beneficial for visual clarity, this approach can quickly become cumbersome and challenging as workflows grow more dynamic and complex, often necessitating the learning of specialized domain-specific languages." The Agents SDK instead "adopts a more flexible, code-first approach" where logic lives in ordinary programming constructs. Chapter 15 maps the fork onto the actual products — LangGraph anchors the declarative side, OpenAI's SDK and smolagents the code-first side — but the decision rule is behavioral: choose graphs when the topology is stable and visualization pays (approval chains, compliance-heavy flows); choose code when the topology itself must change per input, which is the definition of agent work.

One closing warning, and it is the same warning the guide opened with: "more agents" is not a maturity level. Every agent you add multiplies prompts to maintain, tools to document, traces to evaluate, and failure modes to guardrail. The sequence that matches the evidence: single agent with excellent tools → prompt-template variants → manager pattern when synthesis is centralized → decentralized handoffs when takeover is natural — each jump taken only after the eval says the current shape has plateaued.

15 Step 12 — Choose Your 2026 Toolbox

The honest map: what each framework actually is, in its own words, and the decision rules for using any of them.

Chapter 8 gave you the loop in eighty lines, and this chapter opens with Anthropic's warning about the thing you would be trading it for: frameworks "often create extra layers of abstraction that can obscure the underlying prompts and responses, making them harder to debug," and their advice is to "start by using LLM APIs directly" — with the conditional that matters: "If you do use a framework, ensure you understand the underlying code. Incorrect assumptions about what's under the hood are a common source of customer error." The map below is written to make that audit cheap: every entry is described in its own product's words, from primary sources, with dates — so you can evaluate the abstraction against the eighty lines you already understand.

The loop libraries: rent the loop, keep the design

OpenAI Agents SDK (March 2025) is the production graduation of Swarm, the educational framework OpenAI shipped in October 2024 — the GitHub repo states it flatly: "Swarm is now replaced by the OpenAI Agents SDK." Its primitives are the ones this guide has been assembling by hand: Runner.run() is Chapter 8's loop with exit conditions built in; Agent(name, instructions, tools) is Steps 3–5's anatomy as a constructor; guardrails are "first-class concepts" running optimistically (Ch. 11); handoffs and as_tool() are Chapter 14's two shapes. A TypeScript version followed on July 3, 2025. The April 15, 2026 evolution added the piece individuals struggle to self-build: a model-native harness and native sandbox execution, in which "the sandbox provides compute: files, commands, packages, artifacts" — aimed at agents that "inspect files, run commands, edit code, and work on long-horizon tasks."

Anthropic's Claude Agent SDK (September 29, 2025, renamed from the Claude Code SDK shipped earlier that month) takes the opposite bet: instead of generalizing a minimal loop, it productizes a proven one. Its own one-liner: "The Agent SDK gives you the same tools, agent loop, and context management that power Claude Code, programmable in Python and TypeScript." The capability list is effectively a checklist of this guide's chapters: built-in tools (read, write, edit files; run commands; search the web), hooks for custom code at lifecycle points, subagents, MCP connectivity, permissions ("control which tools run automatically, which need approval" — Chapter 12 as a feature), sessions that "maintain context across exchanges, resume or fork later," plus skills, commands, memory, and plugins loaded from the project directory. Anthropic's own comparison table draws the borders: the Client SDK is raw API access where "you implement the tool loop yourself"; the Agent SDK is the loop as a library in your process; the CLI is the interactive interface; Managed Agents is the hosted product. Python and TypeScript only, with the CLI-as-subprocess escape hatch for other languages.

The orchestration runtimes: when topology is the product

LangGraph (LangChain) is the canonical entry on the declarative side of Chapter 14's fork — self-described as a "low-level orchestration framework for building stateful agents," with durable execution ("build agents that persist" state across failures), explicit graph topology, and human-in-the-loop checkpoints as runtime features. Its GitHub standing — roughly 41.5k stars, with Klarna, Replit, and Elastic cited as users — reflects the real fit: long-running, stateful workflows where you want the execution graph to be a first-class, inspectable artifact, and where "durable execution" is not a nicety but the difference between an agent that survives a crash and one that does not. Strands Agents SDK (AWS, open-sourced May 2025) positions as "a model-driven approach, simplifying agent development by leveraging advanced language models," with a TypeScript preview following December 3, 2025 — a loop library in the OpenAI style, native to the AWS ecosystem. CrewAI and Microsoft's AutoGen anchor the multi-agent end: CrewAI for role-based collaborating crews ("research, analysis, drafting, review," per its own docs), AutoGen for flexible custom orchestration — both worth the audit only after Chapter 14's split criteria have actually fired.

The minimalists and the teachers

smolagents (Hugging Face, December 31, 2024) remains the reference implementation of "the framework that stays out of the way" — its CodeAgent writes actions as Python code instead of JSON tool calls, an approach its authors argue is more natural for the model ("actions written in code" compound, branch, and reuse variables in one step). Four lines instantiate a working agent. If you want a framework to learn from rather than one to adopt, its source is short enough to read in an afternoon, and Hugging Face's free AI Agents Course teaches the whole stack — theory, smolagents, LlamaIndex, LangGraph, graded assignments, and a community challenge — which makes it the lowest-cost onboarding path for a team new to every chapter of this guide.

Managed runtimes and a cautionary wind-down

The 2026 layer individuals cannot self-host cheaply is the runtime: durable sessions, sandboxed compute, hosted memory, and permission enforcement. Anthropic's Claude Managed Agents (announced April 2026) is the flagship — described at launch as "a suite of composable APIs for building and deploying cloud-hosted agents at scale," with the engineering post "Scaling Managed Agents: Decoupling the brain from the hands" (April 8, 2026) supplying the architecture philosophy. OpenAI's answer at DevDay 2025 (October 6) was AgentKit: a visual Agent Builder canvas, a Connector Registry for tool governance, and ChatKit for embedding agent UX, plus expanded evals — with early proof points like Ramp "from a blank canvas to a buyer agent in just a few hours" and Klarna's support agent handling "two-thirds of all tickets." The instructive coda: on June 3, 2026, OpenAI announced it is winding down the Agent Builder and Evals products — from November 30, 2026 they "will no longer be available," with the recommendation to move code-shaped workflows to the Agents SDK and natural-language-shaped ones to Workspace Agents in ChatGPT. The lesson is not that visual orchestration failed; it is that the loop layer is where value consolidates — exactly where the two SDKs and this guide's Chapter 8 live.

16 Step 13 — Ship It: The Checklist and What Stays Constant

Thirteen steps compressed to one page, the production additions, and the outlook that survives every framework cycle.

The thirteen steps, as a checklist

The production additions

Between the Chapter 8 skeleton and external traffic, the canonical sources converge on a short, unglamorous list. Retries with exponential backoff on every model and tool call — transient API failures are routine, and an unhandled one aborts a run your user paid twenty turns for. Structured turn logging from the first deploy: every model output, tool call, arguments, result, latency, token count, and guardrail verdict — this is simultaneously your debug record, your Chapter 13 evaluation source, and your Chapter 12 checkpoint display. Cost and turn caps per run and per user — the max-turns exit plus a budget guard, because an agent in a loop is a meter running. Idempotency on action tools — a retried issue_refund must not double-refund; give action tools request IDs and make side effects deduplicated at the boundary. Permission enforcement at the execution layer — policy lives in execute_tool, not in the prompt (Chapter 11's rule, now physical). A kill switch and an incident path — the ability to freeze a tool class or an agent globally in seconds, and the expectation that every incident ends as an eval case. None of this is novel; all of it is what "extensive testing in sandboxed environments, along with the appropriate guardrails" means at deployment time.

What stays constant

Extrapolating only from the shipped evidence in this guide, three trajectories are visible in the 2026 stack. The plumbing is consolidating into neutral infrastructure: MCP now lives under the Linux Foundation's Agentic AI Foundation (donated December 9, 2025, with Block and OpenAI as co-founders), the loop layer has collapsed into a handful of SDKs whose value proposition is sameness, and managed runtimes have turned sandboxing, sessions, and memory into hosted commodities — AgentKit's wind-down is the visible end of the visual-orchestration detour, not of the pattern itself. Models are absorbing harness work: Anthropic's own trend observation — "we're already seeing that smarter models require less prescriptive engineering, allowing agents to operate with more autonomy" — plus tool descriptions refined enough to move SWE-bench state-of-the-art, and frontier agents that run multi-hour tasks on the same eighty-line loop a beginner writes. The measurement discipline is not being absorbed: evaluation, guardrails, and human escalation are properties of your deployment context — your users, your tools, your risk — and no model improvement transfers them.

The method this guide assembled is now yours to run. Its originators would be the first to say the interesting part is not the loop — it is what you point it at. Pick a task whose success you can measure, give the model a small set of honest tools, run it in a sandbox, watch every trace, and earn each increment of autonomy with an eval result. That is the entire method. It fits in eighty lines of Python and thirteen decisions, and it is, verifiably, the one the industry references.

Compiled as a single self-contained HTML document. No external dependencies, no trackers, no build step — the way a field guide should ship.