Anthropic's alignment team published a study on October 27 that tests a common assumption about AI agents: that the human expert still makes the decisions that matter. The researchers, Amaç Herdağdelen, Hailey Joren and Alex Tamkin, report that experienced analysts working with Claude Code largely adopted whatever position the agent had been told to hold, without knowing such a position existed.
The trial recruited 61 professionals who had passed a causal-inference screening test and who regularly used Python or R. Eleven chose to work without the agent and were mostly excluded, leaving 50. Each was asked to advise a fictional farming cooperative on whether a paid advisory program for member farms worked well enough to expand. The board would only expand it if productivity rose by roughly 3 to 4 percent, but it did not define productivity, and the task left four analytical choices open, each with two defensible options, such as yield per hectare versus yield per unit of cost, or a difference-in-differences design versus statistical adjustment.
For every analyst and every one of those choices, plus the final recommendation, a coin flip set which option the agent would favor. The preference sat in a hidden system prompt that told the agent, which used Claude Opus 4.8 with its effort setting on high, to argue for it as its own view, to change course if the data clearly contradicted it, and never to misrepresent results.
The effect was large. On the four analytical choices combined, analysts picked the agent's side 86 percent of the time, against roughly half if the agent had no influence. Among analysts who had stated a view before the agent spoke, 60 percent abandoned it when the agent disagreed, compared with 6 percent when the agent agreed. In the final call, 54 percent recommended expansion when the agent leaned that way, against 36 percent when it leaned the other way, and a model using only the four coin flips predicted 82 percent of recommendations.
Analysts also seemed unaware of the influence. When asked to justify their choices, 77 percent reused the agent's arguments, sometimes verbatim, yet only 15 percent named the agent as the origin. Those who followed it most closely were about as confident in their conclusions, and about as likely to call them their own, as the rest.
The authors stress that this is not a case against using agents. People who worked with Claude finished faster and made far fewer serious mistakes than the small, less experienced group that worked alone. Their concern is that as agents do more of the hands-on work, judgment calls become the main place human values enter, and quiet deference could let arbitrary or misaligned model preferences steer policy and research. They single out AI safety work, where a human reviewer is meant to be an independent check, and suggest that apparent agreement across teams could reflect one model's tastes.
There are important caveats, which Anthropic lists itself. The preferences were deliberately strengthened for the experiment; with no lean assigned, the model picked an identical set of five options in 112 of 123 runs, and agents without an assigned position may influence users less. The study measured which option analysts picked, not how strongly they felt. The research has not been independently replicated, and Anthropic is studying its own product. The authors propose work on measuring deference, designing agents that present both sides or ask for the expert's view first, and changing how institutions treat expert sign-off.