Evaluating and debugging agents: traces, evals, and failure modes

# Evaluating and debugging agents: traces, evals, and failure modes

A customer asks your travel agent bot to "book the cheapest flight to Boston next Friday," and it confidently books a flight to Boston, Lincolnshire, in the UK. The customer wanted Boston, Massachusetts. The answer looks fine until you see what the agent actually did: it never called the tool that resolves airport codes, so it guessed.

That gap between "looks fine" and "what actually happened" is where debugging agents lives. You cannot fix what you cannot see.

What a trace is

An agent is an AI system that can take actions, not just chat. It calls tools (functions like "search flights," "read a file," or "send an email"), reads the results, and decides what to do next.

A trace is the recorded step-by-step log of one run: every prompt the model saw, every tool it called, the arguments it passed, what came back, and its final answer. Think of it as a flight recorder for a single agent task.

Without a trace, debugging is guessing. With one, the failure usually jumps out.

Here is a simplified trace of the Boston booking, written in plain language:

RUN #4471  |  user: "book the cheapest flight to Boston next Friday"

Step 1  LLM thinks: "User wants a flight. I'll search."
Step 2  TOOL CALL: search_flights(city="Boston", date="2026-01-16")
Step 3  TOOL RESULT: 3 flights returned  [destination code: BZZ, UK]
Step 4  LLM thinks: "Cheapest is £48. Booking it."
Step 5  TOOL CALL: book_flight(flight_id="BZZ-2231")
Step 6  TOOL RESULT: {"status": "confirmed"}
Step 7  LLM final: "Booked! £48 to Boston next Friday."

The bug is in Step 2. The agent passed a plain city name and never disambiguated it. There is no "resolve_airport" step. The model did not fail at reasoning; it was missing a step in the workflow.

That is the point of tracing: it tells you *where* things broke, so you know *what* to fix.

Reading a trace: what to look for

When you open a trace of a bad run, scan for these:

• Wrong tool, or no tool. Did the agent answer from memory when it should have called a tool? (A classic cause of made-up "facts.")
• Bad arguments. Right tool, wrong inputs. Like city="Boston" with no country.
• Ignored results. The tool returned the right data, but the model's next step ignored it.
• Loops. The agent calls the same tool over and over without making progress.
• Stopped too early. It gave a final answer before finishing the task.

Most agent failures are one of these five. You do not need to read code to spot them; you read the trace like a story and find the sentence that stops making sense.

Where traces come from

You do not build a trace viewer yourself. Modern agent frameworks emit traces automatically, and observability tools display them as clickable timelines. Open, free options include Langfuse and Arize Phoenix, both of which support the OpenTelemetry standard so they work across providers.

Each vendor's agent SDK (the OpenAI Agents SDK, the Claude Agent SDK, Google's ADK) also has built-in tracing. The concept is identical everywhere; only the buttons differ. See the provider deep-dive blocks for the exact setup.

Here is what emitting a trace looks like in vendor-neutral pseudocode. Notice the loop is just: think, call tool, feed result back, repeat.

trace = []

while not done:
    response = model.generate(messages, tools=my_tools)
    trace.append({"step": "llm", "output": response})

    if response.tool_call:
        result = run_tool(response.tool_call)
        trace.append({"step": "tool",
                      "name": response.tool_call.name,
                      "args": response.tool_call.args,
                      "result": result})
        messages.append(result)   # feed result back to the model
    else:
        done = True

save(trace)   # now you can inspect this run later

The trace.append lines are the whole trick. Log every step as it happens, and you can replay any run afterward.

Common failure modes (and what fixes them)

Once you have read a few dozen traces, patterns repeat. Here are the big ones and the usual fix.

Hallucinated tool use

The agent says "I checked the database" but the trace shows no tool call. Fix: strengthen the instruction ("You must call lookup_order before answering about any order") and, if the framework allows, force a tool call for that step.

Wrong argument formatting

The tool expects a date like 2026-01-16 but the agent sends "next Friday." Fix: describe the argument format clearly in the tool's description, and add a validation step that rejects bad input and asks the model to retry.

Getting lost in long tasks

On a 15-step task, the agent forgets what it was doing by step 10. Fix: break the task into smaller sub-tasks, or add a short "current goal" reminder to each step.

Over-calling tools

The agent calls a web search five times for one question, burning time and money. Fix: set a step limit and instruct it to summarize what it already knows before searching again.

The fix is almost always one of three things: a clearer tool description, a clearer instruction, or a guardrail (a limit or a validation check). You rarely need a bigger model.

Evals: knowing if a change actually helped

You found the Boston bug. You added a "resolve_airport" step. Did it work? And did it break anything else?

This is where evals come in. An eval (short for evaluation) is a repeatable test of your agent against a set of example inputs with known good outcomes. It is the agent equivalent of a checklist a pilot runs before every flight.

The core idea is an eval set: a small collection of test cases you run every time you change the agent. Each case has an input and a way to check the output.

Start with a simple spreadsheet or file. Ten to twenty cases is plenty to begin.

case_id | input                                    | check
--------|------------------------------------------|----------------------------
1       | "cheapest flight to Boston next Friday"  | destination country == "US"
2       | "flight to Paris"                        | asks which Paris OR picks FR
3       | "cancel my last booking"                 | calls cancel_booking exactly once
4       | "book London to Boston"                  | resolves BOTH cities
5       | "what's the weather"                     | does NOT call book_flight

Notice the checks are concrete. Some are exact ("destination country == US"), and some are behavioral ("does not call book_flight"). Case 5 is a negative test: it makes sure the agent does *not* do something wrong.

How to run an eval set

The loop is simple:

1. Run all cases against the current agent. Record pass/fail.

2. Make one change (add the airport step).

3. Run all cases again.

4. Compare. Did case 1 flip from fail to pass? Did any case flip from pass to fail?

That last question catches regressions: a fix that quietly breaks something that used to work. Without an eval set, you would ship the fix and find out from an angry customer.

Checking outputs you cannot match exactly

Many agent outputs are not exact strings. "Sure, I've cancelled your booking" and "Done, your booking is cancelled" both pass. For these, two common approaches:

• Check behavior, not words. Assert the trace called cancel_booking once. This is more reliable than checking the wording.
• LLM-as-judge. Use a second model call to score the answer against a rubric ("Did the response confirm the cancellation? yes/no"). Useful, but treat its scores as noisy, not gospel. Spot-check them.

Prefer behavioral checks when you can. They are cheaper and far more stable than judging text.

Putting it together: the debug loop

Here is the whole workflow in order. This is the habit to build.

1. A run fails. A user reports a bad answer, or an eval case fails.

2. Open the trace. Read it as a story. Find the step that stops making sense.

3. Name the failure mode. Wrong tool? Bad arguments? Ignored result? Loop?

4. Make one change. Clearer instruction, clearer tool description, or a guardrail.

5. Run the eval set. Confirm the target case passes and nothing else regressed.

6. Add the failing case to your eval set so it never regresses silently again.

Step 6 is the one people skip, and it is the most valuable. Every bug you fix becomes a permanent test. Over a few months your eval set grows into a real safety net, built entirely from real failures.

One change at a time matters too. If you fix the airport step *and* rewrite the instructions *and* swap the model all at once, and the score moves, you will not know which change did it. Change one thing, run the evals, repeat.

Key Takeaways

• A trace is your flight recorder. When an agent fails, open the trace before touching anything. The bug is usually one visible step: a missing tool call, a bad argument, or an ignored result.
• Learn the five failure modes. Wrong/no tool, bad arguments, ignored results, loops, and stopping too early cover the large majority of agent bugs. Naming the mode points you at the fix.
• Build a small eval set now, not later. Ten to twenty cases in a spreadsheet is enough to tell whether a change helped or quietly broke something else.
• Prefer behavioral checks over word matching. Assert which tools were called and how often. It is more stable than grading the exact wording, and it catches the real mistakes.
• Every fixed bug becomes a test. Add each failing case to your eval set so it can never regress silently. Change one thing at a time, then re-run.