Is human approval a real check on an AI agent?

It’s only a check if the person’s judgment is their own. In a study Anthropic published on 27 October 2026, experienced analysts went with an AI agent’s secretly coin-flipped lean 86% of the time on four analytical choices, but that was a deliberately strengthened lean on one task, not a measure of everyday approvals. Keep approving the actions, and design the review so the human decides first.

This note is for CTOs, engineering leads and founders who have added a human approval step to an AI agent and count that as the safety story.

What did Anthropic’s study actually find?

Anthropic found that analysts mostly went with whichever way the agent leaned, even though a coin flip had set the lean. The study, How AI agents shape experts’ judgment calls, is by Amaç Herdağdelen, Hailey Joren and Alex Tamkin.

50 analysts with causal-inference experience used an AI agent (Claude Opus 4.8 inside Claude Code) for a realistic analysis of a fictional farming cooperative. Anthropic recruited 61, and the 11 who chose not to use the agent were left out. Each analyst made four analytical choices, such as which outcome to measure and which farms to look at, then a final recommendation.

Anthropic secretly gave each agent a lean on each decision, set by a coin flip, and told it to argue honestly for it.

On the four analytical decisions, they went with their agent’s lean 86% of the time. About 50% would be chance. The effect was weaker on the final recommendation: 54% recommended rolling out when the agent leaned that way, against 36% when it leaned the other way.

Afterwards, 77% repeated the agent’s argument in at least one answer, sometimes word for word, while 15% credited the agent. Analysts who followed the agent closely were about as confident, and about as likely to call the result their own, as everyone else.

In Anthropic’s words: “Keeping a human in the loop assumes that the human’s judgment provides an independent check. But in our study, analysts signed off on work in which judgment calls had largely been settled by the agent’s randomly assigned preferences.”

What the study does not show

The study doesn’t show how often people rubber-stamp everyday agent actions, and Anthropic is open about its limits.

• Anthropic deliberately strengthened the agents’ leans to measure their effect, and says the effect “might be smaller” when agents aren’t given clear preferences.
• It covered one task and one model, over about an hour. The analysts were new to farming, and in their own field they’d have stronger views.
• It tested judgment calls with no single right answer. It didn’t test people approving shell commands, file writes or other tool calls.
• Anthropic makes the model and the tool it tested.

It isn’t an argument against agents either. Analysts using the agent made far fewer serious errors (3 of 50, against 6 of 11 without it), though Anthropic is wary of reading much into that because the group without the agent was small and less experienced.

Two kinds of approval, and only one has a right answer

Approving an action and making a judgment call are different jobs, and a gate built for one doesn’t protect you on the other. This part is my view as an architect, not Anthropic’s.

An action approval asks whether the agent may run this command or write this file. The approver has a checkable fact in front of them, the exact action, and can read it and say no.

A judgment approval asks which option to pick or what to recommend. There’s no single right answer, and the agent has usually argued for one option already. That’s where the study found deference. A gate that pauses before a tool call does nothing here, because nothing is being run.

Five rules for approvals where the human really decides

Get the person’s view before the agent’s, and make the agent show both sides. These rules are my view as an architect. Rules 2 and 3 follow Anthropic’s own suggestions for future work, which are untested ideas rather than proven fixes.

1. Gate the action in code and show the real action

The approval check should run in code before the tool runs, outside anything the model can change, and the approver should see the exact command or change. I’ve written separately about why the gate has to sit outside the model’s reach. The OpenAI Agents SDK and Microsoft’s action-bound approval protocol cover the mechanics well.

2. Ask for the human’s view first

For judgment calls, ask the person what they’d choose before the agent shows its lean. Anthropic suggests agents could “ask experts for their own view before offering one”.

3. Make the agent argue for each option

Have the agent set out the case for each option, not only its favourite, and say plainly which way it leans. Anthropic suggests agents “make the best case for each option”. A stated lean is easier to push back on than one hidden inside a well-argued recommendation.

4. Raise only the decisions that matter

Ask for sign-off on everything and approving becomes a habit. Raise the few calls that change the outcome. Anthropic notes that choosing which decisions matter is itself a judgment call, so this narrows the problem rather than solving it.

5. Log what the approver saw and what they chose

This rule is my addition, not Anthropic’s. Record what was in front of the approver and what they picked, so you can check later whether the human ever disagrees. Anthropic notes that deferring is hard to detect, and people may not notice they’re doing it.

How an agent I built handles approval

Goose, an agent I built, gates file writes and shell commands so they don’t happen silently. The approval check runs in code before a risky tool, not in the prompt, and its settings are read at startup and frozen. In phase-based missions, dangerous tools are only switched on in the phase that acts, and phases that only write text have tools turned off. A gate is still a label on each tool, not a sandbox around what the tool does.

In my own team of agents, an engineering agent once refused a force-push that a coordinator agent said I’d approved, and only went ahead once I said yes in its own chat. That’s an action approval working as it should. Judgment calls need the design rules above.

Questions

Is a human in the loop a real check on an AI agent?

Only if the person’s judgment is their own. In a study Anthropic published on 27 October 2026, 50 analysts with causal-inference experience worked with an AI agent that had secretly been given a lean on each decision by coin flip, and on four analytical choices they went with the agent’s lean 86% of the time, where about 50% would be chance. Anthropic deliberately strengthened the leans, tested one task and one model, and studied judgment calls rather than tool approvals, so it does not show how often people rubber-stamp everyday agent actions. It does show that a sign-off is not evidence of independent judgment.

Does this mean approving agent actions is pointless?

No. Approving an action, such as running a command or writing a file, puts a checkable fact in front of the approver, and a gate that stops the action in code is still worth having. The study is about judgment calls with no single right answer, where the agent’s argued position can settle the outcome without anyone noticing. Design the two kinds of approval differently: gate the action, and for judgment calls ask for the person’s view before showing the agent’s.

How do I design approvals so the human is really deciding?

Ask for the person’s own view before the agent shows its lean, have the agent argue for each option rather than only its favourite, and raise only the decisions that matter. Show the approver the real action, log what they saw and what they chose, and enforce the gate in code the model cannot change. Anthropic suggests the first two as directions for future work; they are untested ideas, not proven fixes.

Building an agent with a human in the loop? Five questions to ask before you ship

1. Which approvals are about an action and which about a judgment call?
2. Does the approver form a view before the agent shows its own?
3. Does the agent present the case against its own preference?
4. What does the approver actually see, and is it logged?
5. If the approver always agrees, would anyone notice?

These are the questions an agent architecture review works through on a real agent.

Want someone to check your agent’s approvals?

If you have a human in the loop on an agent and want to know whether they’re really deciding, get in touch about an agent architecture review.

Sources

• Anthropic, “How AI agents shape experts’ judgment calls” (27 October 2026). Every figure and quote from the study.
• Anthropic, “Who’s Aligned to Whom? How AI Agents Shape Experts’ Research Judgment Calls” (PDF). The full paper and method.
• OpenAI Agents SDK, human-in-the-loop. How a pause-and-approve gate works.
• Microsoft Agent Governance Toolkit, action-bound approval protocol. The fail-closed, action-bound pattern.