Most enterprises shipping agents have written one reassuring sentence into their governance policy: a human approves consequential actions. Most enterprises deploying AI agents have written a reassuring sentence into their governance policy: a human approves consequential actions. The sentence is easy to write and hard to make true.
I build forward-deployed agents for a living. I can tell you where that sentence breaks. It breaks on the fortieth approval of the day, when a tired operator between two other tasks clears the queue by reflex. The audit log records a human decision. No decision happened.
The data says the gate is already gone
This is not a hypothetical. Blanket approval fails: according to Anthropic, users approve 93% of Claude Code permission prompts, a pattern Anthropic describes as approval fatigue. That number alone should end the debate about whether a universal approval step is a safety guarantee.
It gets worse when you measure what the human actually catches. In Anthropic’s testing, human reviewers caught 13.6% of harmful actions. Put those two figures side by side and the picture is stark. You have a reviewer who approves almost everything and catches almost nothing. That is your least reliable component, and you have placed it at the exact point where you believe risk is contained.
The mechanism is old and well understood. Article 14 of the EU AI Act requires that overseers of high-risk systems remain aware of the possible tendency of automatically relying or over-relying on system output, known as automation bias. Aviation and medicine named this decades ago. Agentic AI is recreating it one confirmation dialog at a time.
A rubber stamp costs more than no gate
Here is the argument I make to executives who think an approval step buys them protection. A degraded checkpoint carries the full cost of a control, in reviewer time and cycle delay, while delivering little of its protection. It also creates an audit trail implying that judgment was exercised, which stops being a defense when an agent takes a damaging action.
Sit with that. You pay for the control. You slow the workflow. You generate a record of human decisions that were never made. Then the agent wires the wrong amount or deletes the wrong records, and your own logs testify that a person signed off.
This is the Cost of Doing Nothing in its sneakiest form. CODN usually shows up as missed automation or slow adoption. Here it shows up as theater you are already funding. One survey of 41 government oversight policies concluded that people are unable to perform the desired oversight functions, and that such policies provide a false sense of security in adopting algorithms and enable vendors and agencies to shirk accountability for algorithmic harms. A control that reports success whether or not it works is a liability with a reassuring name.
Design gates that survive volume
The fix is engineering, and it starts by cutting prompt volume at the source. Gate the actions that are hard to undo and visible outside the company: payments, external messages, permission changes, unrecoverable deletes. Let the reversible work run. Gating everything causes a specific harm: volume causes fatigue and fatigue causes rubber-stamping, so a gate on reads actively degrades the gate on refunds. Gate the irreversible, let the rest run.
Then measure the humans, because the audit log will lie to you. Watch two numbers together. A 99 percent approval rate with a 1.8 second median is a log with a delay in front of it, and it is worse than having no gate at all, because it produces a record of human decisions that were never made and a false sense that the risk is handled.
Instrument the queue adversarially. Seed 1 to 2% of the queue with actions you know are wrong, and if reviewers approve them, you have measured the rubber stamp directly. Keep sessions short, because the vigilance drop shows up inside 15 minutes. Give the reviewer a consequence in plain language, so the screen reads “refund $4,200 to an account opened 3 days ago” instead of raw JSON.
Let the system around the model carry load the human cannot. Anthropic’s example design stops the model and escalates to a human after 3 consecutive denials or 20 total denials in a session. Even its automated classifier has limits: Anthropic reports a 17% false-negative rate on real overeager actions in its auto mode pipeline, and states the approach is not a drop-in replacement for careful human review on high-stakes infrastructure. So you layer: automated controls absorb volume, pattern triggers surface anomalies, and scarce human judgment concentrates where both are needed.
The market is drifting the wrong way while this gets solved. Six months ago, 40% of organizations required a human to review high-risk AI actions before they happened; today that number has dropped to 25%, and full agent autonomy for high-risk actions more than doubled from 11% to 26%.
Stop treating the approval button as the answer. Build the loop that measures its own pulse. The teams that instrument the human the way they instrument the model will scale agents safely. Everyone else is signing a logbook on the way to the post-mortem.