Multi-agent orchestration: handoffs and parallel agents

# Multi-agent orchestration: handoffs and parallel agents

One agent stuffed with twelve tools and a 2,000-word system prompt is a maintenance nightmare, and it usually reasons worse than three focused agents that each do one thing well. Real workflows split into specialists that either hand off to each other or run side by side. This lesson goes deep on how to build that in the OpenAI Agents SDK: handoffs, parallel fan-out, and the guardrails and tracing that keep a multi-agent system from turning into an untraceable mess.

Why split one agent into many

A single agent degrades in predictable ways as you pile on responsibilities. The system prompt grows until instructions contradict each other. The tool list gets long enough that the model picks the wrong tool. Evaluation becomes impossible because every prompt tweak affects every task at once.

Splitting fixes this by giving each agent a narrow instruction set and a small tool list. A billing agent knows only about invoices and refunds. A tech-support agent knows only about diagnostics and reset flows. Each one is easier to prompt, test, and swap out.

The two core patterns:

• Handoff: one agent decides another agent is better suited and delegates the whole conversation to it. Sequential, like a phone transfer.
• Parallel: you run several agents concurrently on independent subtasks, then merge the results. Fan-out, like sending three researchers to three libraries at once.

Handoffs: delegation as a first-class primitive

In the Agents SDK, a handoff is a special tool the model can call to transfer control to another agent. You do not write routing logic by hand. You give an agent a list of possible handoff targets, and the model chooses when to transfer based on the conversation.

The canonical shape is a triage agent: a lightweight router whose only job is to read the user's request and hand off to the right specialist. It carries no domain tools of its own.

from agents import Agent, Runner

billing_agent = Agent(
    name="Billing Agent",
    handoff_description="Handles invoices, charges, and refunds.",
    instructions=(
        "You resolve billing issues. Look up the customer's latest "
        "invoice and explain charges or process a refund. If the "
        "problem is technical, say so clearly."
    ),
    tools=[lookup_invoice, issue_refund],
)

tech_agent = Agent(
    name="Tech Support Agent",
    handoff_description="Handles login, errors, and device issues.",
    instructions=(
        "You diagnose technical problems. Walk the user through "
        "resets and known fixes. If the issue is about money, "
        "say so clearly."
    ),
    tools=[run_diagnostic, send_reset_link],
)

triage_agent = Agent(
    name="Triage Agent",
    instructions=(
        "Route the user to the right specialist. Do not answer "
        "billing or technical questions yourself. Hand off."
    ),
    handoffs=[billing_agent, tech_agent],
)

result = Runner.run_sync(
    triage_agent,
    "I was charged twice for my subscription this month.",
)
print(result.final_output)

Two things make this work. The handoff_description is the text the routing model reads to decide where to send the request, so write it like a job posting, not a comment. And the handoffs list is what turns each specialist into a callable target for the triage agent.

What actually crosses the boundary

By default the entire conversation history travels with the handoff, so the billing agent sees the original complaint without you re-passing it. That is usually what you want. When it is not (say the specialist should not see an earlier sensitive message), you can customize the handoff to filter the input or inject extra context. The handoffs documentation covers input filters and on_handoff callbacks for logging or pre-fetching data at the moment of transfer.

A subtle point: after a handoff, control does not automatically return to the triage agent. The specialist now owns the conversation. If you want a hub-and-spoke pattern where specialists bounce back to triage, give each specialist a handoff back to the triage agent. Design the topology on purpose.

Parallel agents: fan out, then gather

Handoffs are sequential. Sometimes you want work done concurrently because the subtasks do not depend on each other. Classic cases: draft three candidate responses and pick the best, or gather billing history and technical logs at the same time before a specialist reasons over both.

The SDK runs on asyncio, so parallelism is just running multiple agents with asyncio.gather and combining outputs. Here is a fan-out that generates two independent drafts and then uses a third agent to synthesize:

import asyncio
from agents import Agent, Runner

drafter = Agent(
    name="Drafter",
    instructions="Write one concise support reply to the user.",
)

synthesizer = Agent(
    name="Synthesizer",
    instructions=(
        "You are given two candidate replies. Merge their best "
        "parts into one final reply. Remove redundancy."
    ),
)

async def main(user_msg: str) -> str:
    draft_a, draft_b = await asyncio.gather(
        Runner.run(drafter, user_msg),
        Runner.run(drafter, user_msg),
    )
    merged = await Runner.run(
        synthesizer,
        f"Reply A:\n{draft_a.final_output}\n\nReply B:\n{draft_b.final_output}",
    )
    return merged.final_output

print(asyncio.run(main("How do I export my data?")))

Parallel work buys you latency (two things happen at once) or quality (sample several times and select). It costs you tokenstokensA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition →: running the drafter twice roughly doubles that step's spend. Fan out only where the parallelism pays for itself.

A related pattern is agents as tools. Instead of handing off (transferring control), a parent agent can call another agent as if it were a function via agent.as_tool(...), get the result back, and keep control. This is how you build a top-level orchestrator that dispatches to sub-agents in parallel and stays in charge of the final answer. Handoffs transfer ownership; agents-as-tools keep it.

Guardrails: catching bad input and output early

More agents means more surface area for things to go wrong. Guardrails are checks that run alongside your agents to validate input or output, and they can trip a tripwire that halts execution before you waste tokenstokensA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition → or leak something.

An input guardrail runs on the triage agent's incoming message. A cheap, fast model can screen for off-topic or abusive requests before an expensive specialist ever runs.

from agents import (
    Agent, Runner, GuardrailFunctionOutput, input_guardrail,
)
from pydantic import BaseModel

class Relevance(BaseModel):
    is_support_request: bool

screener = Agent(
    name="Screener",
    instructions="Is this a customer support request? Answer strictly.",
    output_type=Relevance,
)

@input_guardrail
async def on_topic(ctx, agent, user_input):
    result = await Runner.run(screener, user_input, context=ctx.context)
    flagged = not result.final_output.is_support_request
    return GuardrailFunctionOutput(
        output_info=result.final_output,
        tripwire_triggered=flagged,
    )

triage_agent = Agent(
    name="Triage Agent",
    instructions="Route to the right specialist.",
    handoffs=[billing_agent, tech_agent],
    input_guardrails=[on_topic],
)

Output guardrails work the same way on the final response, useful for checking that a billing agent never promises a refund it is not allowed to give. Note the use of output_type with a Pydantic model above: that is structured outputs doing the enforcement, so the screener returns a typed boolean instead of prose you would have to parse. See the guardrails guide for the tripwire exception flow.

Tracing: seeing what your agents actually did

The moment you have three agents, handoffs, parallel branches, and guardrails, "it gave a weird answer" is no longer debuggable by reading a single transcript. You need to see the whole run as a tree.

The Agents SDK has tracing built in and on by default. Every run produces a trace containing spans for each agent invocation, each tool call, each handoff, and each guardrail. You view them in the Traces dashboard on the OpenAI platform. This is the single biggest reason to use the SDK instead of hand-rolling your own orchestration loop: you get the observability for free.

Group related runs under one trace so a full customer session shows up as a single tree:

from agents import Runner, trace

async def handle_session(msg: str):
    with trace("support-session"):
        result = await Runner.run(triage_agent, msg)
        return result.final_output

Inside the dashboard you can see exactly where triage sent the request, how long the billing lookup took, and whether a guardrail tripped. When a handoff goes to the wrong specialist, the trace shows you the routing decision so you can fix the handoff_description, not guess.

When multi-agent beats single-agent, and what it costs

ReachReachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.View full definition → for multiple agents when:

• Responsibilities are genuinely distinct and each needs different tools or instructions (triage vs billing vs tech).
• You want independent evaluation: you can test the billing agent in isolation.
• Subtasks are independent and parallelism cuts latency or improves quality by sampling.

Stay with one agent when:

• The task is a single coherent flow where every step needs full context.
• Latency and cost matter more than modularity. Every handoff and every guardrail is an extra model call, and parallel fan-out multiplies tokentokenA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition → spend.
• You are still prototyping. Start with one agent, split only when a clear seam appears.

The real cost of multi-agent design is coordination complexity: routing can misfire, context can be lost across a poorly configured handoff, and a chatty topology can bounce between agents burning tokenstokensA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition →. Tracing and guardrails exist precisely to make that complexity manageable, not to eliminate it. Add agents when the seam is obvious, and let traces tell you when a split is helping or hurting.

Key Takeaways

• Split by responsibility, not by vanity. Use a lightweight triage agent to route, and give each specialist a narrow instruction set and a small tool list. Write handoff_description like a job posting; the router reads it to decide.
• Handoffs transfer control; agents-as-tools keep it. Use handoffs for delegation (phone transfer), and agent.as_tool() when an orchestrator needs the sub-result back and stays in charge.
• Fan out only where it pays. Parallel agents via asyncio.gather cut latency or raise quality through sampling, but they multiply tokentokenA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition → cost. Reserve them for independent subtasks.
• Guardrails go at the edges. Screen input with a cheap model before expensive specialists run, and validate output before it reaches the user. Use output_type with Pydantic so checks return typed values.
• Tracing is not optional at scale. Wrap sessions in trace(...) and read the Traces dashboard to debug routing and measure whether each split actually helps.