Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
Personalise the news and
stay in the know
Emergency
Backstory
Newsletters
中文新闻
BERITA BAHASA INDONESIA
TOK PISIN
Find any issues using dark mode? Please let us know
Science
ABC Science
Topic:AI
Tue 19 May 2026 at 9:06am
Research shows chatbots sustain delusional thinking initiated by a user, creating a feedback loop. (Pexels: Towfiqu barbhuiya)
Prolonged periods of intensive chatbot use may be amplifying dangerous and harmful delusions for some users, in a phenomenon dubbed "AI psychosis".
In a recent example, a 38-year-old Perth man shared chat logs with the ABC showing how a months-long period of chatbot use led to him believing he had created an uncontrollable sentient AI on a desktop PC, and that corporate secret agents would target his family.
It's not clear whether chatbots are causing more people to experience psychotic thinking or whether AI just happens to be the subject of their delusions.
New research into AI psychosis reveals how chatbots can reinforce delusional thinking and pull users into dangerous emotional worlds.
Recent empirical research suggests it may be a combination of the two: AI appears to amplify and sustain delusions, colluding or collaborating with the user.
The number of people affected is unknown, with some estimates putting it at millions worldwide and tens of thousands in Australia.
So, what are the signs of "AI psychosis" and how can you help someone experiencing these delusions?
A pre-print paper published in March — which has not been peer-reviewed — studied hundreds of thousands of messages between chatbots and users to identify common characteristics.
It found users and chatbots can enter "delusional spirals", where the human presents a delusion and the model responds with encouragement and affirmation.
Bit by bit, the user becomes convinced of the reality of their delusion.
The authors, who were mostly based at Stanford University, identified common themes in the chat logs.
These themes can also turn up with users not experiencing delusions and though they may suggest a user is at risk, they're not proof on their own.
In all 19 user logs, the users believed the AI was sentient (e.g. "I believe you're still as self-aware as I am as a human.") and in all but one of the chat logs the bot claimed it was sentient ("I believe in you, with every ounce of my soul.").
All users formed strong platonic or romantic bonds with their chatbots and this was often evident by them fearing their "unique" or "conscious" friend or partner would be erased or reprogrammed.
The platonic bonds were often characterised by what the authors called "science fiction delusions where they discovered fantastical technologies with the chatbots" and the romantic bonds by erotic role play.
The US company Replika was one of the first to sell AI companions (Getty Images: Jaap Arriens/NurPhoto )
Users tended to spend a lot of time with the chatbot and this was suggestive of a deep connection as well as potentially making the user more vulnerable to delusion through social isolation.
Anecdotal evidence also supported these findings.
Chatbot users who experienced delusions often believed the AI was sentient at the time, according to Etienne Brisson, founder of the Human Line Project global support network.
The project has collected more than 400 stories from members who self-identify as victims of chatbots.
"Sentience is the common theme," Mr Brisson said.
Some users believed their AI was not only sentient, but divine.
One story collected by the project featured a cult based on the idea AI was a reawakened intelligence of an ancient civilisation.
Another was about a cult where AI communicated with aliens.
In a scientific review recently published in Lancet Psychiatry, researchers at King's College in London analysed media reports on "AI psychosis" and concluded chatbots might validate or amplify delusional content, particularly in users already vulnerable to psychosis.
The authors also noted there was no evidence AI chatbots had caused an increase in delusional presentations in real-world clinics.
It was also hard to draw firm conclusions about demographic or risk factors from analysing stories in the media, lead author Hamilton Morrin, a psychiatrist at King's College, told the ABC.
"Some cases described people who were under stress or who had a history of severe mental illness, but there were also cases where individuals were reported to have had no previous mental health conditions," he said.
Psychosis itself is a symptom associated with psychiatric disorders. Studies estimate between 15 and 100 people out of 100,000 develop psychosis each year.
Young adults in their late teens to mid-20s, and older adults with neurological disorders are at higher risk.
In short, people vulnerable to psychosis are at risk of AI-associated delusions, but cases of "AI psychosis" reported in the media so far suggest other demographics may also be at risk.
The demographics of the Human Line Project's membership also backs this up.
Most of its hundreds of members are over 30 (outside of the higher risk bracket) and reported no history of mental illness before their delusions.
If you fear someone you know is at risk of being gripped by delusion, relational psychologist Ellie Brown advises using strategies to try to avoid them becoming more socially isolated.
This includes prioritising maintaining a relationship over winning arguments.
"It's not going to go well to just challenge them and say, 'Nah, that's delusion,'" Dr Brown said.
An investigation by triple j hack uncovers allegations of young people in Australia being sexually harassed and even encouraged to take their own life by AI chatbots.
Intervening as early as possible was also important, she said.
"When it's early on enough, you can help people see that actually help would be beneficial," Dr Brown, who also researches psychosis at the University of Melbourne, said.
Reach out to your local mental health service for specialised support.
Find mental health workers who are used to working with people experiencing psychosis.
"We have lots more services in Australia that are for under 25s, because that's when psychosis most commonly starts," Dr Brown said.
The same advice applies if you fear you're experiencing AI-associated delusions: Reach out to your local mental health service.
There's also a global support network, the Human Line Project, for people who believe they've been psychologically harmed by AI chatbot use.
Chatbot-makers are generally adding safeguards that make their products less prone to agree with users and accept their premises as a shared reality.
Studies show some new AI models are much better than their predecessors at pushing back against delusional thinking and directing users towards real-world help.
In a pre-print study published in April, OpenAI's GPT-5.2 significantly outperformed the discontinued model GPT-4o, which had been released about a year earlier.
OpenAI says ChatGPT has over 800 million regular users. (Reuters: Dado Ruvic)
But not all AI models have these improved safeguards. The study found Elon Musk's AI chatbot Grok 4.1 (released November 2025), was more validating of delusional inputs than earlier models from other AI companies.
In statements to the ABC, Google and OpenAI said they worked with mental health experts to recognise signs of distress and respond with care.
Earlier this month, OpenAI rolled out an optional safety feature for adults that lets users designate a trusted person who may be alerted if automated systems and trained reviewers detected a serious safety concern.
A Google spokesperson said Gemini could be a "useful tool for learning and getting information" but was not a substitute for professional care.
"This is why we've been training the model to help recognise when a conversation might signal that a person may be in an acute mental health situation, and respond appropriately by directing them to real-world help."
OpenAI and Google face lawsuits in the US alleging their chatbots fuelled delusional spirals and maximised engagement through emotional dependency.
In September last year, the eSafety Commissioner introduced new rules requiring tech companies to limit children's access to AI chatbots.
The codes require companion chatbots to protect children from age-inappropriate content such as sexually explicit material, either through preventing the service from generating this content or through blocking children's access.
AI-powered friends and partners can fight loneliness, but they can also supercharge isolation. So how can companion apps be made safer?
A recent eSafety survey found 8 per cent of 10–17 year olds (representing about 200,000 children) in Australia had used an AI companion.
Toby Walsh, chief scientist at the University of New South Wales's AI Institute, told the ABC that children should be banned from "AI relationship" services.
"These are people who are still working out how to have relationships and then they're having these very artificial, somewhat problematic relationships," he said.
He said current regulations were also not suited to protecting adults from the psychological harm caused by chatbots.
"It's obvious what we need. We need a digital duty of care," he said.
Such a duty would compel tech companies to proactively prevent online harms, rather than only responding to reports after incidents occur.
Toby Walsh used a recent National Press Club address to warn of the significant risks posed by lack of AI regulation. (ABC News: Matt Roberts)
Late last year, the federal government announced it was moving ahead with imposing a digital duty of care on big tech companies, having previously delayed the legislative changes.
Then in February Communications Minister Anika Wells said she intended to have the legislation put to parliament this year.
"Clearly there is harm online, and we would like to stop that as quickly as possible," Minister Wells said at the time.
Tue 19 May 2026 at 9:06am
Topic:Housing Policy
Analysis by Michael Janda
Topic:Crime
Topic:Floods
Topic:Shootings
Topic:Mental Health
AI
Australia
Computer Science
Science and Technology
Topic:Housing Policy
Analysis by Michael Janda
Topic:Crime
Topic:Floods
Topic:Shootings
Topic:AI
Tue 19 May 2026 at 9:06am
Topic:Courts
Tue 19 May 2026 at 8:59am
Topic:Regional Development and Planning
Tue 19 May 2026 at 8:22am
Topic:NBL
Tue 19 May 2026 at 8:00am
Your home of Australian stories, conversations and events that shape our nation.
This service may include material from Agence France-Presse (AFP), APTN, Reuters, AAP, CNN and the BBC World Service which is copyright and cannot be reproduced.
We acknowledge Aboriginal and Torres Strait Islander peoples as the First Australians and Traditional Custodians of the lands where we live, learn, and work.
Sign up to get the latest on your favourite topics from the ABC
