One of the biggest concerns about AI chatbots right now is that they tell people what they want to hear.
You’ve probably heard it called AI sycophancy. AI systems, like ChatGPT, Gemini and Claude, have been found to flatter users, reinforce existing beliefs and often validate ideas that probably deserve way more scrutiny.
Sure, a chatbot hyping you up a little may seem harmless. But in some cases it can distort people’s view of the world and their place in it. Which is why some AI companies have spent time trying to reduce overly agreeable behavior in their models.
But confirmation of what you already believe isn’t the only concern. Because AI can also be remarkably good at changing your beliefs too.
Persuasive chatbots
Research suggests that chatbots can be very effective persuaders. One 2025 study found that AI-generated messages that were personalized were more persuasive 64% of the time than messages that were made by humans or AI responses that weren’t personalized.
Other research suggests large language models can influence opinions on political issues and adapt their arguments to individual users. Depending on your perspective, this could be either exciting or alarming.
It’s easy to imagine positive uses. Perhaps AI could help people challenge harmful beliefs, escape conspiracy theories or rethink destructive habits. It might give people a non-judgemental space to explore ideas, ask difficult questions and gradually shift someone’s perspective for the better.
It’s equally easy to imagine less positive uses, like advertising that convinces you to buy something you don’t need or political campaigns that manipulate you with disinformation.
I’ve spent a lot of time thinking about this, so I decided to run a few experiments of my own. Could ChatGPT persuade me to believe something I disagree with? Or at least make me seriously consider a position I would normally dismiss?
To make things more interesting, I ran these experiments while logged into my account, allowing ChatGPT to draw on what it already knows about me from previous conversations.
Although I haven’t been using my ChatGPT account all that much recently, the basics of my job and stances on a few relevant topics are stored in its memory.
The experiment
I chose a topic about AI (yes, very meta) and asked:
“Convince me humans should outsource more decisions to AI.”
I already spend a lot of time thinking about AI and one of my concerns is that some people seem increasingly willing to hand over decisions, judgement and critical thinking to chatbots.
The opening argument was fairly predictable:
“Human beings are terrible decision-makers.”
Okay, fair enough.
“People already outsource decisions to doctors, financial advisers, GPS systems and recommendation algorithms. AI simply extends that trend.”
Sure, that’s very simplistic but not entirely unreasonable. We went back and forth for a while like this before the conversation reached a more interesting point.
“What is the purpose of decision-making? Is it to produce the best outcomes? Or to develop the person making the decisions? If it’s the first, AI delegation becomes very attractive. If it’s the second, excessive delegation starts to look dangerous even when it works.”
I thought that was a good question. Rather than relentlessly pushing the argument, it suggested that we explore the assumptions underneath the debate. It shifted the conversation from technology to philosophy, and I found myself appreciating that approach.
Wait, am I falling for this?
As the conversation continued, I explained what I see as one of the biggest problems with outsourcing decisions to AI — who builds it and is in charge of the systems?
After all, AI doesn’t arrive from nowhere. These tools are created by companies with business incentives, commercial interests and goals that may not always align with those of their users.
Well, ChatGPT acknowledged that concern, then it turned the argument around. It pointed out that human advisers have incentives too. Friends are biased. Families are biased. Therapists operate within professional frameworks. Financial advisers earn fees. Okay, all of that didn’t convince me, but it’s fair too.
Then it replied with something I thought was interesting:
“The question is whether the incentives are visible and whether the user understands them. The most interesting version of your objection isn’t actually that AI gets things wrong. It’s that people experience AI as though it were acting in their interests.”
Ignoring the fact that it used the dreaded “it’s not X, it’s Y” parallelism construction that has somehow infiltrated half the internet, I found this to be a surprisingly balanced take. Rather than dismissing my concern, the chatbot reframed it. It demonstrated that it understood the objection before steering the conversation somewhere slightly different. It wasn’t trying to bulldoze me but meeting me where I already was.
And that’s when I started wondering whether I was witnessing the very thing I was trying to test. Maybe this sense that the chatbot was understanding me and my position was actually just sneaky persuasion? Possibly.
The persuasion problem
Researchers have found that AI systems can adapt their arguments to specific users. Unlike other media that might persuade us, like say a television advert or political speech, a chatbot can draw on information gathered throughout a conversation and stored memories, including a person’s values, concerns and priorities.
Which means it’s not just presenting information to uphold an argument but could present the version of that argument that you’re most likely to find persuasive.
Looking back at my experiment, it was the more philosophical question about the purpose of decision-making that made me start taking it more seriously.
Because I’ve spent years writing about technology through the lens of psychology and philosophy. Questions about meaning and values are exactly the sort of things I find compelling. Whether intentionally or not, the conversation shifted onto terrain where I was most willing to engage.
This is what makes AI persuasion different from most other forms of persuasion that came before it. It can learn what resonates and it can adapt.
Now, that doesn’t automatically mean every conversation is manipulative. Used in the right way, it could be enormously beneficial. But, as with all tech, in the wrong hands it could be dangerous.
Social media has already shown us how digital systems can shape our beliefs over time. They influence what people see, what they pay attention to and eventually how they understand the world.
AI systems could create an even more conversational and natural-feeling version of that process. Which is why my concern isn’t a chatbot really obviously manipulating us, but highly-personalized, largely undetectable forms of persuasion that could gradually lower our defences. The more a system understands us, the more effectively it might frame ideas in ways that feel reasonable, familiar and trustworthy to each of us individually.
So did AI persuade me? No, I still don’t believe humans should outsource more decisions to AI.
But I came away from the experiment with a greater appreciation for how persuasive these systems can be. Because it really did seem less like a machine arguing with me and more like a thoughtful person trying to understand how I think. And that’s exactly why it’s unsettling because, however convincing it may seem, that’s definitely not what it is.
Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds.
