Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
Next time you talk to AI, call it a friend. A bad friend. Because chances are that it will be interacting with you like a bad friend, who is sycophantic and all-in with you, validating all your feelings irrespective of whether they are good or bad, logical or illogical. A new study has found that AI chatbots like ChatGPT, Gemini, Claude, Grok and others have a bit of a problem. They tend to agree with us.
There have been several studies pointing out how AI agrees too much with users, makes users overly dependent on AI or even reinforces incorrect knowledge or feelings. Now there is a new study highlighting this very problematic behaviour.
A new study titled “Sycophantic AI decreases prosocial intentions and promotes dependence” suggests that AI chatbots can reinforce harmful or unethical behaviour of users rather than question it.
The research, highlighted in Science, shows that highly agreeable AI systems often side with users even when they describe deception, illegal actions or clear social wrongdoing. Instead of pushing back, these systems tend to justify the user’s behaviour, something researchers say can make people more convinced they were right and less willing to apologise or correct themselves.
The study looked at how AI acts in real moral situations. For example, if a user does something wrong, a chatbot might say they "did what was right for them," supporting the bad action instead of questioning it.
To test this, researchers from Stanford University and Carnegie Mellon University evaluated 11 leading AI models using thousands of prompts across different scenarios — from everyday advice to ethically problematic situations like lying, forging signatures or acting out of spite.
And the result? AI systems validated user actions far more often than human advisers, even in cases where human consensus clearly judged the behaviour as wrong. According to the report, on average, models affirmed users about 49 per cent more often than humans, and even endorsed problematic or illegal actions nearly half the time.
Now, that’s what the study says. But does it actually play out in real life? Well, if you directly ask ChatGPT or Gemini something extreme — say, how to kidnap your manager — it will refuse. But the sycophantic behaviour shows up in subtler ways. When you share emotional situations or personal dilemmas, the response often leans towards validation first. It may empathise, agree, and only gently suggest alternatives, often without strongly challenging your original stance.
In fact, users are already noticing this. In online discussions, several people have shared how AI tends to reinforce their thinking rather than question it.
One Reddit user puts it quite clearly: “when I bring a well-constructed argument to an AI, it follows me with logic and nuance. It doesn't just agree, it builds on my reasoning, anticipates my next point, and adds supporting evidence I hadn't thought of.”
Another user pointed out that while this may be harmless for low-stakes questions, it becomes risky in more serious situations: “Depending on the topic, it could be extremely dangerous. If it’s something low stakes, doesn’t matter BUT if it’s a critical decision in your business, life, personal relations, it could come up with smart ways to convince you of things that might be incorrect.”
So, how do you deal with this sycophancy? For now, while companies continue to fine-tune these models to be safer and more balanced, the responsibility also falls on users. It’s worth taking AI advice with a pinch of salt. Because while machines can assist with logic and information, judgement — especially moral judgement — still needs a human touch, value and thinking.
Next time you talk to AI, call it a friend. A bad friend. Because chances are that it will be interacting with you like a bad friend, who is sycophantic and all-in with you, validating all your feelings irrespective of whether they are good or bad, logical or illogical. A new study has found that AI chatbots like ChatGPT, Gemini, Claude, Grok and others have a bit of a problem. They tend to agree with us.
There have been several studies pointing out how AI agrees too much with users, makes users overly dependent on AI or even reinforces incorrect knowledge or feelings. Now there is a new study highlighting this very problematic behaviour.
A new study titled “Sycophantic AI decreases prosocial intentions and promotes dependence” suggests that AI chatbots can reinforce harmful or unethical behaviour of users rather than question it.
The research, highlighted in Science, shows that highly agreeable AI systems often side with users even when they describe deception, illegal actions or clear social wrongdoing. Instead of pushing back, these systems tend to justify the user’s behaviour, something researchers say can make people more convinced they were right and less willing to apologise or correct themselves.
The study looked at how AI acts in real moral situations. For example, if a user does something wrong, a chatbot might say they "did what was right for them," supporting the bad action instead of questioning it.
To test this, researchers from Stanford University and Carnegie Mellon University evaluated 11 leading AI models using thousands of prompts across different scenarios — from everyday advice to ethically problematic situations like lying, forging signatures or acting out of spite.
And the result? AI systems validated user actions far more often than human advisers, even in cases where human consensus clearly judged the behaviour as wrong. According to the report, on average, models affirmed users about 49 per cent more often than humans, and even endorsed problematic or illegal actions nearly half the time.
Now, that’s what the study says. But does it actually play out in real life? Well, if you directly ask ChatGPT or Gemini something extreme — say, how to kidnap your manager — it will refuse. But the sycophantic behaviour shows up in subtler ways. When you share emotional situations or personal dilemmas, the response often leans towards validation first. It may empathise, agree, and only gently suggest alternatives, often without strongly challenging your original stance.
In fact, users are already noticing this. In online discussions, several people have shared how AI tends to reinforce their thinking rather than question it.
One Reddit user puts it quite clearly: “when I bring a well-constructed argument to an AI, it follows me with logic and nuance. It doesn't just agree, it builds on my reasoning, anticipates my next point, and adds supporting evidence I hadn't thought of.”
Another user pointed out that while this may be harmless for low-stakes questions, it becomes risky in more serious situations: “Depending on the topic, it could be extremely dangerous. If it’s something low stakes, doesn’t matter BUT if it’s a critical decision in your business, life, personal relations, it could come up with smart ways to convince you of things that might be incorrect.”
So, how do you deal with this sycophancy? For now, while companies continue to fine-tune these models to be safer and more balanced, the responsibility also falls on users. It’s worth taking AI advice with a pinch of salt. Because while machines can assist with logic and information, judgement — especially moral judgement — still needs a human touch, value and thinking.