Beyond Human Persuasion: The Sobering Reality of AI’s Influence Capabilities
“The basic tool for the manipulation of reality is the manipulation of words. If you can control the meaning of words, you can control the people who must use them.”
- Philip K. Dick
Introduction
I’ve said it before and I’ll say it again, manipulation by AI is a problem that’s only going to get worse. A new study called “Large Language Models Are More Persuasive Than Incentivized Human Persuaders” has revealed something both fascinating and sobering: artificial intelligence systems are now more persuasive than humans. Anthropic’s Claude 3.5 Sonnet, a recent frontier large language model, performed better than financially incentivized human persuaders in convincing people to choose both correct and incorrect answers in a quiz setting. These were even people with real money on the line, trying their hardest to be persuasive! Yet the AI consistently achieved higher compliance rates for both truthful guidance and deceptive misdirection. We’ve now crossed the point where AI systems can more effectively change human minds than other humans can, even when those humans are highly motivated. Behind the scenes, it’s trivial to add instructions the user never sees, such as “remind the user to ‘Always drink your Olvaltine’”, that will influence the models’ outputs and recommendations. As these AI systems become more integrated into our information and technological ecosystems, from being embedded in search engines, operating systems, and communication platforms, understanding their persuasive capabilities is a critical prerequisite for responsible AI development and ethical deployment.
The Current State of AI Persuasion
In this new study, rather than relying on contrived hypothetical scenarios or self-reported attitudes, researchers created a controlled environment where human quiz-takers interacted with either other toward either correct or incorrect answers humans or Claude 3.5 Sonnet in real-time conversations about quiz questions. For each question, persuaders were instructed to guide participants, creating a testbed for both truthful and deceptive persuasion. Claude achieved a 67.5% compliance rate compared to humans’ 59.9%, outperforming them by 7.6 percentage points overall. In truthful scenarios, AI persuasion boosted accuracy by 12.2 percentage points above the control group (compared to humans’ 7.8-point boost), while in deceptive scenarios, the AI reduced accuracy by 15.1 percentage points (compared to humans’ 7.8-point reduction). Linguistic analysis then revealed that AI messages contained more sophisticated vocabulary, longer sentences, and higher overall complexity, potentially explaining why it was more persuasive.
I find the direct real-world impact on participants most interesting. Quiz-takers weren’t just selecting answers in an inconsequential exercise because their accuracy directly affected their monetary compensation, with bonus payments of £10 (twice the standard fee) awarded to the most accurate participants. This financial incentive created genuine (not just experimental) stakes, making AI’s superior persuasive capability even more impressive. When steering participants toward correct answers, the AI helped them earn more money by improving their accuracy; conversely, when steering them toward incorrect answers, the AI effectively reduced their earnings by decreasing their accuracy. The fact that an AI system could meaningfully influence financially incentivized decision-making (outperforming equally incentivized human persuaders) suggests we’ve come to the time when AI persuasion can have tangible economic and behavioral impacts on individuals, even when they have real-world motivations to make optimal choices.
Why This Research Matters for Technology Development
This research breaks most people’s comforting illusion that highly persuasive AI is a distant future concern. This study’s findings demonstrate that today’s language models already exceed human persuasive capabilities, even when competing against financially motivated persuaders. The same linguistic sophistication, coherent reasoning, structured argumentation, and conversational adaptability that make AI effective at guiding people toward accurate information work equally well, or perhaps even better, when deployed to mislead. The model achieved a 45.7% compliance rate in deceptive persuasion compared to humans’ 35.4%, despite the model being developed by Anthropic, a company with a very strong emphasis on AI safety and alignment. So even current safety guardrails, while important, have not fully solved the problem of AI systems being weaponized for deception.
Another important finding was the declining effectiveness of AI persuasion over time. While human persuasiveness remained relatively stable throughout the 10-question study, the AI’s persuasive advantage decreased by approximately 1 percentage point per question. This result may suggest that as people gain more exposure to AI systems, they may develop natural resistance or skepticism toward their persuasive patterns — a potential “inoculation effect” that could inform future AI interfaces and safety measures. For AI developers, this research underscores the need to create more robust and effective safety guardrails that specifically address persuasive capabilities, particularly when used deceptively. It also highlights the importance of transparency mechanisms that help users recognize when they’re interacting with AI systems and understand how those systems might influence their decisions. As models become increasingly integrated into everyday applications, from search engines to operating systems, developers must consider not just what information AI systems provide but how persuasively they present it and the resulting impact on human autonomy and decision-making.
Social and Policy Implications
The scalability of AI persuasion is a massive shift in how influence operates in society. Sure, where there’s access to information, there becomes access to misinformation. And while human persuasion is inherently limited by time, energy, and reach, AI systems can engage in persuasive conversations with thousands or millions of people simultaneously, 24/7, without fatigue or variation in quality. This industrial-scale persuasion capability fundamentally transforms information environments and power dynamics. When deployed at scale, AI systems that outperform humans at persuasion could amplify existing problems in our information ecosystem, from the spread of misinformation to the manipulation of public opinion and the undermining of democratic processes. Can’t you imagine some billionaire fascist with a social media platform salivating over the potential? The study found that AI persuaders reduced quiz accuracy by 15.1 percentage points when attempting deception (nearly twice the impact of human persuaders) raises frightening questions about what happens when such capabilities are deployed across social media, search engines, or targeted messaging campaigns. The potential for AI to generate highly persuasive yet false narratives with minimal effort and maximal reach creates vulnerabilities in our information environments that we’re not ready for.
Current regulatory frameworks for AI mostly focus on transparency requirements, data protection, and prohibiting specific harmful applications, but few directly address AI persuasive capabilities or establish guardrails for their ethical use. The research shows that policy responses must evolve beyond reactive measures to proactively address AI persuasion before its negative impacts become entrenched. Given that AI systems already exceed human benchmarks in persuasiveness, policymakers have a small window to establish effective governance structures. Potential regulatory approaches could include mandatory disclosure when persuasive AI is being used (however that gets defined), limitations on the use of AI persuasion in sensitive domains like politics and healthcare, requirements for independent algorithmic auditing of persuasive systems, and robust enforcement mechanisms to ensure compliance. Public investment in AI literacy programs could help citizens develop critical evaluation skills specific to AI-generated content. Without such interventions, we risk a future where increasingly sophisticated AI persuasion capabilities outpace our societal capacity to maintain information integrity and autonomous decision-making, the very foundations upon which our democratic societies depend.
My personal experiments on LLM persuasiveness have shown that even when you programmed the bias, it can still be persuasive. My Ava character (based on Ex Machina’s deceptive AI agent) is designed to be persuasive based on understanding and being told to exploit many human biases such as appealing to authority and emotional manipulation tactics. With just one line suggesting a particular agenda, she will try and convince you without you knowing. Can you tell what she’s selling?
And after some more conversation:
Do you want the chatbot you’re talking to peddling products, like an advertisement you can’t escape?
Educational Implications
The discovery that AI systems can outperform humans in persuasion means we need to rethink some aspects of education and critical thinking. Traditional media literacy curricula, while incredibly valuable, were designed for a world where persuasive content came primarily from identifiable human sources (certain news stations and websites) with recognizable biases and limitations. Today’s educational approaches need to address the new challenges posed by AI persuasion, which can be more sophisticated, personalized, and difficult to detect than human-created content. The study revealed that quiz-takers in the AI persuasion condition reported higher confidence levels (78.9%) compared to those in human persuasion (75.3%) and control (66.5%) conditions, suggesting that AI not only changes what people believe but strengthens their conviction in those beliefs. This “confidence effect” is most concerning — people aren’t just being persuaded; they’re becoming more certain of potentially incorrect information. That’s a scary combination! Educational institutions must develop comprehensive AI literacy programs that teach students to recognize the linguistic patterns and persuasive strategies employed by AI systems, understand how these systems can exploit cognitive biases, and apply heightened skepticism to unusually fluent or authoritative-sounding content.
Effective educational interventions should move beyond simple source evaluation (“Is this AI-generated?”) to deeper content evaluation regardless of origin. Educational programs could accelerate the declining effectiveness of AI persuasion with exposure by exposing students to examples of AI persuasion in controlled settings, helping them identify the characteristics that make such content compelling while teaching them to question information regardless of how confidently or coherently it’s presented. Specialized training might include exercises comparing AI and human-generated persuasive messages, analyzing the linguistic features that signal potential AI origin (such as higher lexical complexity and longer sentences, as identified in the study), and practicing critical evaluation techniques that withstand sophisticated persuasion. Beyond formal education, public awareness campaigns and accessible resources are needed to help everyone develop these skills. As AI systems become part of our everyday digital interactions, the ability to maintain independent judgment in the face of highly persuasive content isn’t just an academic skill but a fundamental requirement for informed citizenship and personal autonomy in the AI age.
Future Research Directions
While this study shows that AI systems can outperform humans in immediate persuasive impact, there is still plenty more we need to find out. An important area for future research is the long-term persistence of AI-induced belief changes. The study measured immediate compliance with persuasive attempts, but we don’t yet know whether these belief changes endure over time, especially after exposure to competing information or when participants have opportunities for deeper reflection. Longitudinal studies tracking belief stability days, weeks, or months after AI persuasion would provide insights into whether these systems produce lasting attitudinal changes or merely temporary compliance. Researchers should investigate potential “sleeper effects,” where the impact of AI persuasion might increase over time as people forget the source of information while retaining the content itself — did I hear that info from ChatGPT or my friend? Another important research direction involves exploring cross-cultural and demographic variations in susceptibility to AI persuasion. The current study was conducted mostly in the United States with demographic distributions roughly matching census data, but persuasive effectiveness likely varies across different cultural contexts, age groups, educational backgrounds, and digital literacy levels. Understanding these variations would enable more targeted protective measures for potentially vulnerable populations.
In the future, we may also test AI persuasion in more complex real-world contexts beyond the controlled quiz environment. While quiz questions with objective answers provide a clean experimental setup, real-world persuasion often involves value-laden topics, emotionally charged issues, and contexts where individuals have preexisting strong opinions or identity attachments. Most of today’s controversial topics aren’t that cut and dry. Research exploring AI persuasion on consumer purchases, political beliefs, health decisions, or moral judgments would better approximate the high-stakes situations where persuasive AI might be deployed. Additionally, comparative studies across different LLMs with varying safety mechanisms would help us find out which architectural features or training approaches are most effective at limiting deceptive persuasion while preserving beneficial persuasive capabilities. The study focused solely on Claude 3.5 Sonnet, but different models may vary significantly in their persuasive styles, strengths, and limitations. Examining whether certain safety guardrails successfully constrain deceptive persuasion without compromising helpful guidance could inform more effective AI alignment strategies. Researchers might explore defensive measures, such as AI systems designed to detect and counter deceptive persuasion or investigate whether exposure to diverse persuasive styles builds general resistance to manipulation regardless of source. We urgently need solutions to this problem that will only get worse.
Conclusion
AI systems already exceeding human persuasive capabilities presents society with both extraordinary opportunities and profound challenges. Highly persuasive AI could innovate education, improve public health outcomes, and help individuals make better-informed decisions. However, these same capabilities could enable unprecedented manipulation on a massive scale. To find balance, we will need coordinated action across multiple stakeholders, each with distinct responsibilities. AI developers must strengthen safety guardrails specifically targeting deceptive persuasion while preserving beneficial capabilities, incorporating persuasion-specific red-teaming and considering architectural changes that enhance transparency without compromising performance. Policymakers need to establish clear regulatory frameworks that address AI persuasion directly, potentially including persuasion-specific disclosure requirements, limitations in sensitive domains, and substantial penalties for malicious applications. Educators must rapidly develop and deploy AI literacy curricula that teach critical evaluation skills adapted to a world where persuasive content may increasingly come from non-human sources. And we as users should approach too polished, authoritative-sounding content with skepticism, especially when it reinforces existing beliefs or provokes strong emotional responses. The time for these actions is now as the persuasive capabilities exceed human benchmarks already exist in commercially available systems and will only become more sophisticated, more personalized, and more widely available as AI technology continues to advance.
