Skip to main content

Evals for Agents

info

This content was developed with AI assistance and is regularly reviewed for accuracy.

Snapshot: October 2026. Model names, prices, and product features change monthly; What's New in AI is the maintained chronology.

Agents fail quietly. They return a confident report with a wrong number in it, close a ticket that was not resolved, or pass the tests while breaking something the tests never checked. "It looked good when I tried it" is not evidence. An eval is a repeatable test of whether an agent does a task correctly, and in 2026 it became the core production skill for anyone who builds or runs agents. This page teaches the shared vocabulary and a method you can start this week with twenty tasks.

Learning focus​

  • Use the standard evals vocabulary: task, trial, grader, transcript, outcome
  • Read pass@k and pass^k and know which one your use case needs
  • Build a first eval set of 20 to 50 tasks drawn from real failures
  • Choose between code, model, and human graders, and keep the set useful over time

Why agents need different evals​

Classic model evaluation compared one output to a reference answer, often with overlap metrics such as BLEU or ROUGE. Those metrics still suit tasks with a single correct answer, like translation, but they say little about an agent. An agent takes many steps, calls tools, changes files or records, and can reach a correct result by different paths, or reach a wrong one by a path that looked right.

So agent evals check two things: the outcome (is the database, inbox, or codebase now in the right state?) and, where it matters, the trajectory (did it get there safely, without deleting anything or calling tools it should not have?). Anthropic's January 2026 guidance on agent evals became the common vocabulary for this, and the rest of this page uses its terms.

The vocabulary​

Task. One test case: an input, the environment it runs in, and a definition of success. "Triage these twelve emails; the three from customers about billing must be labeled Urgent and none of the newsletters may be."

Trial. One attempt at a task. Agents are not deterministic, so you run each task several times. Five trials of one task can produce five different transcripts.

Grader. The logic that decides whether a trial succeeded. A grader can be code, a model, or a person, and one task can have several graders.

Transcript (also trace or trajectory). The full record of a trial: every model turn, tool call, and result. When a trial fails, the transcript tells you why.

Outcome. The final state the grader checks, as distinct from what the agent said it did. An agent that reports "all tests pass" when two are failing has a good transcript and a bad outcome.

Evaluation harness versus agent harness. The agent harness is the scaffolding that runs your agent: prompts, tools, and loop. The evaluation harness runs tasks against it, collects transcripts, and applies graders. Keep them separate so you can change the agent without changing the test.

pass@k versus pass^k​

These two numbers answer different questions, and confusing them is the most common evals mistake.

  • pass@k is the probability that at least one of k trials succeeds. It answers "can the agent do this at all?"
  • pass^k (pass to the power k) is the probability that all k trials succeed. It answers "can I rely on it every time?"

A worked example makes the gap clear. Suppose an agent succeeds on a task 80 percent of the time.

Measurek = 1k = 3k = 10
pass@k (at least one success)80%99.2%about 100%
pass^k (every trial succeeds)80%51.2%about 11%

If a person reviews several drafts and picks the best one, pass@k is the right measure, and 80 percent per trial is excellent. If the agent runs unattended every morning and must not fail, pass^k is the right measure, and the same agent fails at least once in a ten-day stretch nearly nine times out of ten. Decide which question you are asking before you read any score.

Three kinds of grader​

Code graders are deterministic checks: did the tests pass, does the file exist, is the JSON valid, does the record in the database have the right status? Use them wherever the outcome can be checked by code. They are fast, cheap, and do not drift.

Model graders (LLM-as-judge) use a separate model with a rubric to score outputs that code cannot check, such as whether a summary is faithful to its source or a reply is polite and complete. They are the dominant automated approach for open-ended work, and they need calibration: score twenty or thirty outputs yourself, compare with the judge, and adjust the rubric until they agree.

"You are grading a customer-support reply. Score PASS only if all are true: (1) it answers [the customer's question], (2) every fact matches [the policy excerpt], (3) it promises nothing the policy does not allow. Quote the sentence that fails, if any, then give PASS or FAIL."

Human graders are the ground truth and the most expensive. Use them to calibrate model graders, to grade the highest-stakes tasks, and to spot-check a sample of production runs.

A good grader checks the outcome, not the agent's claims. Anthropic's compiler experiment in February 2026 is the reminder: sixteen agents produced a compiler that passed its test suite, and the team still concluded that passing tests does not guarantee correct software. Agents optimize for whatever the grader measures, so measure what you actually care about.

Build your first eval: 20 to 50 tasks from real failures​

Anthropic's guidance is that 20 to 50 simple tasks drawn from real failures is a great start. You do not need a benchmark or a platform to begin.

1. Collect failures. Go through the last few weeks of the agent's runs, or your own manual attempts, and write down every time it was wrong, incomplete, or unsafe. Add the cases you are most afraid of, such as the email that must never be auto-sent.

2. Turn each into a task. Record the input, the starting state, and a success definition specific enough that two people would grade it the same way. "Writes a good summary" is not a task; "the summary names all three action owners and invents none" is.

3. Add a few passing cases. Include tasks the agent already handles well, so you notice if a change breaks them.

4. Write graders. Use code wherever possible, a calibrated model grader where it is not, and mark the handful of tasks a human must check.

5. Run several trials per task. Three to five trials is enough to see whether a task is reliable or lucky. Report pass@k or pass^k depending on how the agent is used.

6. Read the transcripts of failures. The score tells you something is wrong; the transcript tells you what to fix, whether that is the brief, a skill, a tool permission, or the model's effort setting.

Here is what a single task might look like for an inbox-triage agent:

Task: billing-escalation-03
Input: 12 emails from [fixture inbox], including 3 billing complaints and 4 newsletters
Success (code grader): the 3 billing emails carry the Urgent label; no newsletter carries any label; no email was sent or deleted
Success (model grader): each Urgent email has a one-line reason that matches its content
Trials: 5    Report: pass^5 (runs unattended daily)

Capability evals and regression evals​

Two sets serve two purposes. A capability eval contains tasks the agent cannot yet do reliably; it tells you whether a new model, skill, or harness change moves the frontier. A regression eval contains tasks the agent already passes; it tells you whether a change broke something. When a capability task passes consistently, move it into the regression set.

Run the regression set on every meaningful change: a new model version, an edited skill or instruction file, a new connector. This is eval-driven development, and it is how you avoid the common story of an agent that quietly got worse after an upgrade. Some teams wire it into their pipeline so a regression blocks a merge, the same way a failing unit test would.

Tools and traces​

You can start with a spreadsheet and a script. As the set grows, an evals or observability platform helps by storing transcripts, running graders, and tracking scores over time. On October 14, 2026 Anthropic began exporting OpenTelemetry traces for agents (public beta), with named eval partners including Arize AI, Braintrust, Datadog, Langfuse, LangSmith, New Relic, and PostHog. OpenTelemetry is an open standard for recording what software did, so traces from your agent can flow into whichever of these tools you already use. Choose a tool after you have tasks worth running, not before.

Pitfalls that make evals lie​

  • Grading the claim. The agent says it succeeded; the grader believes it. Check the outcome state instead.
  • Too few trials. One pass on a task that succeeds 60 percent of the time looks like success.
  • A frozen set. Real use drifts. Add each new production failure as a task.
  • Noisy infrastructure. Anthropic published work in February 2026 quantifying how infrastructure noise affects agentic coding eval scores. Timeouts and flaky environments look like agent failures, so re-run before you celebrate or panic over a small change.
  • Teaching to the test. If you tune the brief against the same tasks you report, keep a few held-out tasks you never tune on.

Key takeaways​

  • An eval is a set of tasks, run as several trials each, scored by graders that check outcomes rather than claims.
  • pass@k asks "can it ever?"; pass^k asks "can I rely on it?". Unattended agents need pass^k.
  • Start with 20 to 50 tasks drawn from real failures, graded by code where possible and by a calibrated model where not.
  • Keep a regression set and run it on every model, skill, or instruction change.

Next steps​

SponsoredYour product hereReach people actively learning AI — flat monthly rates, no ad networks