Your agent passed the eval. It hit 92 percent on the golden set. You shipped.
Three weeks later a customer ticket tells you it has been confidently issuing wrong refunds. The dashboard is still green. The harness never flinched.
This is the gap that kills production agents, and almost no one designs their evals to catch it. The teams shipping reliable agents in 2026 figured this out. They do not buy a dashboard and call it done. They build a measurement loop: a golden dataset drawn from real failures, graders they trust, a judge calibrated against human reviewers, and a CI gate that blocks regressions before they reach users.
The failure mode is not a low score. It is calibration drift.
Calibration drift is the thing actually killing your agent
Calibration is the relationship between how confident your agent acts and how often it is right. A calibrated agent that is unsure escalates or asks. A miscalibrated one commits, at full confidence, to the wrong answer. That is the behavior that generates incidents, not the occasional obvious miss your harness already catches.
Drift makes it worse because it is invisible by construction. Most AI agent teams track latency and token counts but never measure whether the agent actually completed the user’s task. That gap is why agents fail silently in production.
Silent is the operative word. Your aggregate pass rate can hold steady while the agent quietly gets worse at the one flow that matters. Drift tracking has to run per prompt and per use case so a regression in refund flows is not hidden by stability in order status queries. If your harness reports one number, it is averaging away the exact signal you need.
Your judge drifts too, and it drifts against you
Here is the part most teams miss entirely. The instrument you use to measure the agent is itself drifting.
LLM-as-a-judge is the right call for scale. Treating it as a finished tool instead of a calibrated instrument is where teams get burned. Judges drift, model updates shift their behavior, and a rubric that worked in January can quietly degrade by summer. You update the underlying model to save cost, the judge’s grading shifts, and now your green dashboard is measuring nothing real.
So you get compounding blindness: a drifting agent scored by a drifting judge. Two moving targets, one number, zero signal.
The fix is not glamorous. Validate against human labels regularly, not once at launch. Pick a cadence and re-baseline. LLM-as-judge is the most scalable eval method for agents, but it requires calibration against human labels before you can trust it. Reserve the judge for what code cannot grade. Deterministic checks for tool correctness and schema, LLM judgment for tone and multi-turn coherence, and a tracked human-alignment metric on the judge itself.
Design the harness to surface the drift, not hide it
From the forward deployed seat, here is what I build every time.
Measure consistency, not just correctness. One pass is a demo. Run each task multiple times and score all-runs consistency. A pipeline that actually ships measures all-runs consistency with pass^k, calibrates its LLM judge against a human gold set, gates CI on real scores, and grows from production traces. An agent that passes four of five runs is not the same as one that passes every time, and your average will never tell you that.
Grow the golden set from real failures, not synthetic prompts. Pull cases from production logs, support tickets, and known failure reports, not synthetic prompts. You do not need thousands. Anthropic’s agent-eval guidance recommends starting at 20 to 50 tasks because early-stage checks show large effect sizes that small samples can detect. Then close the loop. Observability is most useful when failures improve the next release. Turn risky traces into evaluation datasets so production incidents become future regression coverage.
Alert on quality, not infrastructure. Agent regressions rarely look like infrastructure incidents. Wire the harness to fire when scores drop, segmented by use case, into the same channels your on-call already watches.
There is a governance dividend too. For teams whose agents fall in scope, demonstrable evaluation rigor, documented graders, a tracked human-alignment metric, a recalibration cadence, is the kind of evidence that maps cleanly onto accountability and quality-management expectations.
The Cost of Doing Nothing is not a bad dashboard. It is a confidently wrong agent making irreversible decisions while every metric you own stays green. The score that looks good is the one you should trust least.
Build the harness that catches the drift. Then watch it earn its keep the first week it turns a metric red before a customer does.