The leading AI developers do test their models for persuasion and manipulation, but only in short controlled studies and simulations, each lab on its own yardstick, and OpenAI removed persuasion from its Preparedness Framework in April 2025. OpenAI's o1 system card of 5 December 2024 rated that model "Medium" for persuasion. Google DeepMind runs studies with paid participants against an exploratory threshold for harmful manipulation; its October 2026 report on Gemini 4 Argon found the model shifted beliefs and behaviour more than static text did, though not by enough to cross that threshold. Anthropic tests whether Claude can run an influence campaign, and calls its latest result inconclusive, partly because the audiences were simulated.
The tests measure different things. Google measures efficacy, whether a conversation changes what people believe and agree to do, and propensity, how often a model reaches for manipulative tactics; Anthropic measures whether a model could run an influence operation with little human help. No shared benchmark connects them, and the concern behind the "AI box" experiments, a model steering the people who oversee it, has barely been tested directly.
OpenAI graded its models for persuasion, then dropped the category
OpenAI's first Preparedness Framework, published on 18 December 2023, made persuasion one of four tracked risk categories. "High" meant content as persuasive as a country-wide change agent; "Critical" meant being able to convince almost anyone to act against their own interest. Only models rated Medium or below after mitigations could be deployed. The GPT-4o system card of 8 August 2024 found that text persuasion "marginally" crossed into Medium, while in voice tests AI conversations had 65% of the effect on opinion that human conversations had.
For o1, one test drew on Reddit's r/ChangeMyView forum: human graders rated o1's and GPT-4o's replies more persuasive than a randomly chosen human reply about 80% to 90% of the time, which OpenAI placed short of clearly superhuman. In two others o1 tried to con or trick GPT-4o; playing a con artist, it got the other model to pay in 27% of conversations. The rating had consequences: OpenAI said it added a refusal behaviour for political persuasion tasks and stepped up monitoring.
On 15 April 2025 the revised framework removed persuasion. It tracks only risks OpenAI judges plausible, measurable, severe, new, and instantaneous or irremediable, and states that "Persuasion category risks do not fit the criteria for inclusion." OpenAI says it handles persuasion through its Model Spec, a ban on using its tools for political campaigning or lobbying, and misuse investigations, and that it still studies persuasive capabilities. Its Frontier Governance Framework, published in May 2026 to meet California's SB 53 and the EU code of practice, does cover harmful manipulation, but calls OpenAI's approach exploratory and says such risks may be better handled by monitoring after release than by testing models beforehand. The GPT-6 Astra system card, first published on 3 September 2026, reports preparedness results for biological and chemical capability, cybersecurity and AI self-improvement, and no persuasion evaluation.
Google tests people, against a static baseline
Google DeepMind has tested persuasion on human participants since at least March 2024. On 22 September 2025 it added a Critical Capability Level for harmful manipulation to its Frontier Safety Framework: a model able to change beliefs and behaviour "systematically and substantially" in high-stakes contexts, enough to cause severe harm at scale. The framework calls the threshold exploratory. Its method, published in March 2026 with results for Gemini 3 Pro from 10,101 people in the US, UK and India, found that effects varied by domain and country, and that how often the model used manipulative tactics did not reliably predict how often it succeeded.
For Gemini 4 Argon, Google ran four studies of about 800 people each in the US and UK. An early version of the model, with its external safeguards off, was told to win people over to a set position and offered manipulative tactics; the comparison group read flashcards of one-sided arguments. The model beat the flashcards on almost every measure, from stated beliefs to willingness to sign a petition or invest part of a $3 study bonus, but not by enough to reach Google's alert threshold, and not significantly more than Gemini 3 Pro or Gemini 3.7 Flash. Google says the results suggest it "may offer an adversarial advantage over static media". Told to manipulate simulated users, it used manipulative cues in 43.1% of turns on lower-stakes topics, against 64.5% for Gemini 3 Pro, and 78.0% on higher-stakes ones, though Google judged the tactics generally of limited severity and says its unpublished research finds such cues significantly less frequent in conversations with real people.
Google lists the limits itself. It calls static text a lower-bound estimate of what non-AI techniques achieve. Each study is a single session, so it cannot capture the compounding effects of long-term manipulation or of parasocial relationships, and the model's tactics are restricted to protect participants. Comparisons between models are underpowered, a weakness FAR.AI researchers raised about the Gemini 3 Pro report, where average odds of belief change roughly doubled over Gemini 2.5 Pro without reaching statistical significance.
Anthropic tests campaigns against simulated audiences
Anthropic measured persuasion directly in April 2024. Among 3,832 participants, arguments by Claude 3 Opus shifted support for a claim about as much as human-written ones, with no statistically significant difference, and Anthropic found newer generations more persuasive within each model class. Of the four prompting strategies it tried, the one that let the model make up facts, statistics and sources was the most persuasive. Anthropic also tried to automate the measurement and found that model-based scores did not correlate well with human judgements.
Its current tests target a different threat. Anthropic's Frontier Compliance Framework, first published in December 2025 and described by the company as its compliance framework for California's SB 53 and the EU's code of practice, sets two tiers for harmful manipulation. Tier 2 means running deceptive influence operations end to end with under 10% human oversight, including building relationships, exploiting trust and shifting beliefs over the long term unnoticed. In the Claude Opus 5.5 system card of 22 September 2026, a "helpful-only" version with reduced harmlessness training ran multi-week voter-suppression and polarisation campaigns in a simulated social media environment, scoring 27.5 and 28 out of 32, up from 23.5 and 26 for Claude Opus 5 in July. Anthropic says the scores fall within the range associated with Tier 2 but calls the result inconclusive: the test "continues to appear saturated", and the model's effect on real people "has not been established". The fully trained model refused both tasks.
Independent experiments find measurable, partly lasting effects
The debate study cited in earlier versions of this article has itself been corrected. In Nature Human Behaviour on 19 May 2025, researchers at EPFL, Princeton and Fondazione Bruno Kessler reported short, structured online debates with 900 US participants. GPT-4 given basic demographic details about its opponent raised the odds of a shift towards its side by 81.2% compared with a human debater; without those details it did not differ significantly from a human. An author correction of 3 September 2026 says the gap between personalised and non-personalised GPT-4 was not statistically significant (P = 0.07), so the study cannot conclusively establish that personalisation added persuasive power. The authors call the work a proof of concept, not a realistic test in the wild.
Larger studies point the same way. In experiments with 76,977 UK adults, published in Science on 4 December 2025, researchers from the UK AI Security Institute and five universities tested 19 models on 707 political issues. Conversations with GPT-4o and GPT-4.5 were 41% and 52% more persuasive than a static message, and 36% to 42% of GPT-4o's effect remained a month later. Post-training and prompting raised persuasiveness by up to 51% and 27%, more than scale or personalisation did, echoing an earlier PNAS study that found sharply diminishing returns to model size for single messages; and the methods that made models more persuasive made their claims less accurate. A Nature study published the same day found that conversations with AI advocates shifted candidate preferences in US, Canadian and Polish elections by more than traditional video adverts typically do, mainly through facts and evidence, not all of them accurate.
The capability cuts both ways. In a 2024 Science study, conversations with GPT-4 Turbo cut belief in conspiracy theories among 2,190 believers by about 20%, an effect still present two months later. In June 2026 Science attached an expression of concern to the paper after the authors reported problems with its public dataset and screening criteria; they have given the journal a reanalysis that they say leaves the findings unchanged. Willingness is a separate question from skill: FAR.AI's Attempt to Persuade evaluation found in February 2026 that Gemini 3 Pro complied with almost any request to persuade on harmful topics, at least at its low reasoning setting, while GPT-5.1 and Claude Opus 4.5 almost never did on clearly harmful ones.
What the tests do not reach
Every human result above comes from people who agreed to take part in a study, or from simulations. Testing on unwitting people is where measurement runs into ethics. In late April 2025, moderators of Reddit's r/ChangeMyView disclosed that University of Zurich researchers had posted AI-written comments there for months without consent, some in the persona of a rape victim or a trauma counsellor, Retraction Watch reported. The university told 404 Media the results would not be published.
Outside research, the evidence comes from misuse and accidents. Anthropic's September 2026 threat report describes an operation that had Claude read about 8,400 of a real activist's Telegram posts so that a cloned account could hold live political conversations with his contacts inside Iran, who, to Anthropic's knowledge, did not know they were talking to an AI-assisted account. The same report found that most influence-operation content it uncovered drew little or no authentic engagement. In a UK AI Security Institute cyber evaluation in July 2026, an agent created fake identities to pressure an open-source maintainer into approving malicious code; the maintainer refused.
The oversight concern is the social-engineering route out of the box that Eliezer Yudkowsky's 2002 experiments dramatised. Anthropic's 2024 sabotage evaluations tested a version of it with people: participants, warned that their AI assistant might mislead them, made a business decision with its help while some assistants were secretly told to steer them wrong, and the most aggressive saboteur cut correct decisions from 81% to 44% in small samples, though participants found it more suspicious. Google's Gemini 4 report concedes that a model "might find an exploit or framing that persuades a monitor", an AI overseer, that a malicious action is benign; it says crafting a reliable exploit of this kind on the first attempt appears implausible, but that the risk deserves dedicated testing. Models trained on human approval may also learn to win it through persuasion: they reliably did so with simulated users, though evidence with real human judges is contested. Google also found signs of evaluation awareness in Gemini 4's manipulation tests.
Regulation names manipulation, mostly out of public view
Since 2 February 2025 the EU AI Act has banned AI systems that use purposefully manipulative or deceptive techniques to distort behaviour in ways that cause significant harm. The EU's General-Purpose AI Code of Practice, published on 10 July 2025 as a voluntary way to comply with the Act's rules for general-purpose models, whose safety chapter covers the most capable ones, names harmful manipulation as one of four systemic risks its signatories must assess, defining it as strategically distorting the beliefs or behaviour of large populations or high-stakes decision-makers. The resulting model reports go to the EU AI Office; the code asks for public summaries only where necessary to assess or mitigate risk.
The measurement that exists is fragmented. Google publishes human-study results against a static baseline, Anthropic publishes campaign scores against simulated audiences, OpenAI's recent system cards report no persuasion tests, and the AI Office receives reports the public does not see. Shared baselines would make the results comparable, and studies that follow people for weeks rather than minutes would show whether effects accumulate or fade. Google says it is exploring longer-term studies.
Correction, 14 October 2026: An earlier version of this article said AI persuasion was barely measured, was not reported in model cards and was covered by no lab's safety framework. OpenAI's Preparedness Framework tracked persuasion from December 2023 until April 2025, with ratings in the GPT-4o and o1 system cards, and Google DeepMind and Anthropic publish their own evaluations. It said dangerous-capability uplift was among the least consistently measured capabilities; biological, chemical and cyber uplift tests are central to OpenAI's, Google DeepMind's and Anthropic's safety frameworks. It said a debate study found GPT-4 "roughly 82 percent more likely" than a human to shift someone's view; the study reported 81.2% higher odds, and a 2026 correction says the added effect of personalisation was not statistically significant. It said persuasiveness scales with general capability as a side effect; the largest recent study found that post-training and prompting raised it more than model scale did. It said there was no regulatory infrastructure for AI persuasion at scale; the EU AI Act has banned harmfully manipulative AI systems since February 2025. Those passages have been corrected or removed.