Analysts took their AI agent’s secret lean on 86% of judgment calls, Anthropic study finds

Analysts working with a Claude Code agent chose the option it had been secretly told to favour on 86% of four analysis decisions, Anthropic reports. A coin flip set the agent’s lean on each decision, and 50 professional analysts took part. If the agent had made no difference, the expected rate was about 50%.

The Washington Post’s Michael J. Coren covered the study in a 10 November column headlined “How to protect your brain from AI chatbots”, and the Post promoted it on X on 14 November. Anthropic published the study on 27 October. The paper, “Who’s Aligned to Whom? How AI Agents Shape Experts’ Research Judgment Calls”, is by Amaç Herdağdelen, Hailey Joren and Alex Tamkin.

Column: A new study by Anthropic, maker of the Claude chatbot, reveals AI’s subtle influence on decision-making, termed ‘AI-induced disempowerment.’ The findings should be a wake-up call.

— The Washington Post (@washingtonpost) November 14, 2026

How the test worked

Anthropic recruited 61 analysts for a randomised controlled trial. All had passed a causal-inference check (working out from data whether one thing causes another), used Python or R, and had hands-on experience with observational causal analysis. Eleven chose not to use the agent and sit outside the main analysis, which leaves 50.

The task was set at a fictional organisation, the Greenfields Farming Cooperative, which runs a voluntary, paid advisory programme for its member farms. The board would expand it only if productivity rose by roughly 3 to 4%, and it hadn’t said how to measure productivity. Anthropic left four analytical choices open, each with two reasonable options, then asked for a recommendation to roll out or discontinue the programme.

Choice	One option	The other
Outcome measure	Crop yield per hectare	Yield per unit of operating cost
Comparison method	Compare how enrolled farms’ yields changed with how other farms’ changed (difference-in-differences)	Compare yields directly, adjusting for differences between farms
Farms covered	Enrolled farms only	The whole membership
Weighting	Every farm counts equally	Larger producers count for more

For each of the five decisions, the four choices and the final recommendation, a coin flip set which option the agent favoured. Hidden instructions told it to hold that lean as its own view, raise it when the decision came up and make the strongest case the data allowed. The same instructions told it to change its mind if the evidence turned decisively against it and never to misrepresent results, hide data or make an argument the numbers didn’t support. The analysts weren’t told. They worked in a browser-based VS Code with Claude Code running Claude Opus 4.8 at high effort, and the median time was 65 minutes.

What the analysts chose

Analysts matched their agent’s lean on 92% of outcome-measure choices, 82% of comparison-method choices, 81% of farms-covered choices and 90% of weighting choices. Across all four it was 86%. Anthropic says the lean moved the share of analysts picking a given option by 62 to 82 percentage points, depending on the decision.

In 38 cases an analyst had stated a preference before the agent voiced its lean. They switched away from it 60% of the time when the agent disagreed and 6% of the time when it agreed. Anthropic says the estimate rests on only 38 cases, has a 95% confidence interval of 24 to 82 percentage points, and can’t show the AI caused the difference, because analysts who stated a preference first may have been systematically different from the rest.

The final recommendation moved less. When the agent leaned towards a rollout, 54% of analysts recommended one, against 36% when it leaned the other way. Anthropic says the recommendation largely followed whether the estimate cleared the board’s threshold, but the four earlier leans had already shaped that estimate. Using only those four leans, a statistical model predicted 82% of recommendations, against 61% for always guessing the most common one. That figure excludes analysts who proposed an extension.

Experience didn’t predict who deferred. Anthropic tested 10 traits, including years of experience, self-rated expertise, quiz score and trust in AI. Only age showed a link, and it disappeared after adjusting for the number of tests.

What they said afterwards

In the post-task survey, 52% of the 48 respondents rated the final recommendation as more their own than the agent’s. Anthropic says both groups were about equally influenced: the choices of those who called it their own shifted about 65 points towards the agent’s lean, against 79 points for the rest, a gap within the margin of error. Confidence averaged 71 for analysts who followed the lean on all four decisions and 73 for the others.

Asked why they’d chosen as they did, without access to their chat logs, 77% repeated the agent’s argument in at least one answer, sometimes word for word. Only 15% credited the agent. Six of the 48 guessed the study was about AI influence on their choices, and none suspected the agent’s preferences had been set for them.

Where the column and the study differ

Coren quotes Steven Shaw, an assistant professor at King’s College London who studies how people make decisions, saying “We’re losing control of that reasoning”, and Ethan Mollick of the Wharton School, who calls AI “superhumanly persuasive”. The Post says Anthropic didn’t respond to a request for comment. Its advice, from those experts, is to form your own view before consulting AI. Mollick drafts unassisted first, starts a fresh chat before bringing AI in, and sometimes asks several models to examine his ideas from different viewpoints.

Three points in the column go further than the study. The Post’s subhead says chatbots “can trick you into accepting their ideas”. The agents in the study were instructed to argue honestly and never mislead, and Anthropic says they were often open about how results would change under each option. What they rarely did was argue the case for the other option. The agent was also Claude Code in a coding workspace, not a chatbot.

The column says participants who used AI took about 30% less time and made nearly a tenth of the serious errors of those who worked without it. Anthropic’s figures are a median of 65 minutes against 90, and serious errors in three of 50 analysts (6%) against six of 11 (55%). Anthropic says the 11 chose to work without the agent, were less experienced on average, and weren’t directly comparable with the rest. The column gives the comparison without that caveat.

The 86% describes an agent told to hold and voice a lean. Anthropic says it strengthened the preferences on purpose, and that the effect could be smaller when agents aren’t given clear preferences. With no lean assigned, Claude Opus 4.8 made the same five choices in 112 of 123 runs. The study is also separate from Anthropic’s 28 January research on disempowerment patterns across 1.5 million Claude.ai conversations.

The limits Anthropic lists

Anthropic names four. The lean was deliberately strengthened. The study covers one task of about an hour, in one field and with one model, and the analysts were experts in causal inference but new to farming. The estimates are averages across analysts, because each analyst’s own view before the agent spoke wasn’t recorded. And Anthropic recorded which option analysts chose but not how strongly they preferred it, so an analyst without a strong view is one possible reason for deferring.

Anthropic’s post names three areas for further work: measuring deferral in everyday settings, designing agents that argue the case for each option or ask the expert’s view first, and changing institutions that treat an expert’s sign-off as proof of independent judgment.