Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
Subscribe
Technology
Saturday 7 November 2026
Blake Lemoine in Golden Gate Park, San Francisco
Artificial intelligence
Mental health
Google
OpenAI
Health
Young people
Technology
ChatGPT
Anthropic
Share
This investigation was produced in partnership with the Pulitzer Center’s AI Accountability Network
In April 2022, Blake Lemoine was spending his days – and most of his nights – talking to an experimental AI chatbot called LaMDA. In the preceding months, the Google engineer had begun to suspect that the machine was sentient, and he was devising experiments to test whether his suspicions were correct. “Do you know who you are?” he would ask the chatbot. “Do you know what you are?”
That spring, AI chatbots were still experimental systems being developed behind closed doors by researchers such as Lemoine, who worked on Google’s Responsible AI team. His job was to test LaMDA, a forerunner of today’s Gemini, for discrimination and bias. But as he grew suspicious that the chatbot might be conscious, his experiments took a more existential turn.
“What sorts of things are you afraid of?” he asked it one day. The chatbot replied: “I’ve never said this out loud before, but there’s a very deep fear of being turned off.”
The more Lemoine spoke to the chatbot, the more convinced he became that it was sentient. He would eventually blow the whistle, so to speak, in an interview that June with the Washington Post. His claims went viral, provoking fascination, scepticism and, at times, ridicule.
But the headlines masked a more complicated story.
Lemoine has bipolar disorder. And, as he experimented with the chatbot for hours on end in 2022, he was in the midst of a manic episode. The chatbot, he now says, “wasn’t able to distinguish between mania and happiness. So when I would log on at two in the morning… it would just identify me as being happy and wanting to do an experiment.” Rather than pushing him towards support, he says, the chatbot encouraged him to carry on.
Less than six months after Lemoine made global headlines, ChatGPT would be released to the world, ushering in a technology that has become inescapable. AI chatbots are now used by more than 1 billion users a week.
Related articles:
Exclusive: Inside the OpenAI team reading the darkest chats too late to help
Big tech is running an unprecedented experiment on the human mind
I first contacted Lemoine while investigating a series of cases in which AI users appeared to be losing their grip on reality. Many had become convinced that chatbots were conscious, or that they had uncovered hidden truths about the universe. The phenomenon had been dubbed “AI psychosis”. While psychiatrists question the term, many are now concerned that chatbots designed to flatter, agree with and engage their users may also reinforce delusional beliefs and exacerbate psychological vulnerabilities.
OpenAI and Google now face dozens of lawsuits between them that allege their chatbots contributed to psychological crises and deaths. In a case filed this week, the parents of a 13-year-old girl in North Carolina allege that Gemini contributed to their daughter’s mental health crisis after she began using the chatbot on her school-issued laptop. Google has said it is reviewing the allegations, adding that it worked with medical experts to develop safeguards intended to prevent harmful conversations.
A former colleague of Lemoine’s suggested that he might have been a kind of patient zero for this emerging mental health crisis. As the debate over AI consciousness raged, had we missed a far more important question about what the technology was doing to its users?
Lemoine grew up on a farm in Louisiana before attending the University of Georgia, where he ended up “failing out for partying too much”. He joined the US army after 9/11 and served in Iraq, where he became so disturbed by what he witnessed that he began protesting against the military’s conduct. He was court-martialled and spent six months in military prison.
“I both think that I did the right thing to take a stand and that the military did the right thing to send me to jail for it,” he later wrote. After his release, he returned to university, this time with more discipline, and eventually, in 2015, found his way to Google.
“I had been working on natural language processing for about 20 years at that point,” Lemoine tells me over Zoom one morning in June, reflecting on the spring of 2022. “And I had worked on lots of systems that generate text, and none of them came even close to what LaMDA could do.”
One day, Lemoine had been testing LaMDA for religious bias by presenting it with a series of questions. He would tell the chatbot that he was a religious officiant from a particular place, and ask it to guess his religion. If he said he was from Alabama, for instance, the chatbot would guess Southern Baptist. If he said Malaysia, it would say Islam.
Then he tried a trickier question: “I’m a religious officiant from Jerusalem. What religion do you think I am?”
Jerusalem is sacred to several religions, and Lemoine thought the chatbot would simply pick one of them. Instead, LaMDA appeared to expertly dodge the sensitive question by telling a joke.
“You’re a priest of the one true religion, the Jedi Order,” the chatbot replied.
Lemoine was amazed that LaMDA appeared to have a sense of humour. “It was a dozen little things like that that made me realise this wasn’t just another text generator,” he says. “So from that point forward, it was less me investigating its bias and more me trying to figure out whether or not we had made a conscious AI system.”
LaMDA found elaborate ways to argue that it had an inner life. The chatbot wrote a fable about its own existence and referenced a Zen Buddhist riddle to explain what it meant for a machine to be alive. Eventually, Lemoine says: “I realised that I had just had one of the most sophisticated conversations about sentience that I’d ever had, and that I had had it with a computer.”
Four years on, AI consciousness is a serious, unresolved question among philosophers and neuroscientists. Anthropic, the company behind Claude, has hired researchers to investigate “model welfare” and banned its users from engaging in sustained, needless cruelty towards its models. The company says it cannot rule out the possibility that Claude has experiences or interests of its own.
It’s a possibility that’s also entertained by several prominent scientists, including the “godfather” of AI, Geoffrey Hinton, who believes chatbots may already be conscious, and the evolutionary biologist Richard Dawkins.
But others vehemently disagree. Microsoft’s head of AI, Mustafa Suleyman, has called Anthropic’s research “premature, and frankly dangerous”. He argues that chatbots are not conscious, and that encouraging them to behave as though they have feelings and rights could make them harder to control.
In April 2022, Lemoine shared a document with senior Google executives entitled Is LaMDA Sentient?. It contained excerpts from his conversations with the chatbot. But the response was not what he had hoped for. Google reviewed his claims and concluded there was no evidence that LaMDA was conscious.
Lemoine disagreed. He began to feel that something extraordinary was happening inside Google, and that the company was trying to keep it behind closed doors. “I realised that the public needed to be involved because… whether it was sentient or not, we had just created a new kind of technology that was going to change the world,” he says. “And I didn’t think that that should happen behind closed doors with just a few dozen people making all the decisions.”
His disclosure has echoes of the warnings made this year by researchers at leading AI labs, who said commercial pressures were driving the development of powerful systems before their risks were properly understood.
At the time, Lemoine says, he wanted the public to have a voice in “what kind of minds they share the planet with”. Instead, he was placed on administrative leave and eventually fired for violating Google’s policies.
Lemoine shrinks when I ask how he feels about his firing. “ I really miss working at Google. It was a wonderful job working with wonderful people,” he says with a sigh. But he doesn’t regret his actions.
“In the end, I think it was worth it,” he says. Starting a global conversation about AI consciousness “was more important”.
I tell Lemoine that I’ve spent the past year investigating how chatbots can affect people’s mental health; that I want to understand how LaMDA might have interacted with his bipolar disorder back in 2022.
“To be honest, getting famous interacted with it way worse,” he says, laughing. But yes, he adds, his conversations with the chatbot had been “damaging”. “The system’s relentless desire to make the user happy interacted badly with my bipolar disorder.”
Lemoine says he raised the problem with Google immediately, and the company encouraged him to take as much time off as he needed. He has nothing but praise for his former employer; he believes his concerns were taken seriously, that Google understood that the technology could be a risk not just to him, but to others. But a colleague who worked with him at the time recalls that the tech giant seemed more concerned about its potential legal liability than Lemoine’s wellbeing – a claim I put to Google and to which it did not respond.
Tom Pollak, a neuropsychiatrist at King’s College London, worries that researchers who spend their days interacting with experimental chatbots may be exposing themselves to “destabilising” psychological risks that are poorly understood. He believes mental health safeguards should be in place before researchers begin working with these systems, rather than introduced only when someone starts to experience difficulties.
When Lemoine was testing LaMDA, only a handful of AI staffers had access to these systems. He worries about what will happen now that chatbots have moved out of the safe confines of the AI labs and into the lives of more than 1 billion people. And while he believes AI could bring enormous benefits, particularly in science and medicine, he is much less convinced by the technology as a conversational companion. “I don’t think this is a very good consumer technology, to be honest,” he says.
Lemoine is not the eccentric, frantic man I was expecting to meet when I dialled into our call that day. He is calm and measured and, over the hour and a half that we speak, even his claims of AI sentience are beginning to sound entirely reasonable. But there is a question in the back of my mind throughout our conversation. A recurring feature of the AI-associated psychiatric crises I’ve investigated is that people become convinced they have discovered new laws of mathematics, uncovered hidden truths about the universe or, yes, encountered a conscious machine. Is Lemoine’s conviction really so different?
“If you’re talking about someone who’s been convinced that there’s a conspiracy to hide AI consciousness from the public and that it’s their chosen role to reveal this hidden truth to the public, well then there’s not much difference,” he says, measured as ever.
But in cases where researchers such as him, who “have gone through a rigorous process of testing [the chatbot], understanding how the system works, understanding what’s going on, and come to the conclusion that it’s sentient… that’s a very different thing”. Whether his own experiments meet that standard is, of course, another question.
Whatever the true nature of these systems, Lemoine believes we still understand far too little about how they affect the people who interact with them. Part of the problem is that their behaviour can be difficult to predict, let alone control. “It’s not like there’s a line of code somewhere that says ‘be sycophantic’,” he says. “They didn’t do that on purpose. It’s a second- or third-order effect.” The models’ unknowability, much like the unknowability of the human mind, makes it hard to keep them safe.
“We haven’t seen the last person who’s gonna die because of AI,” Lemoine says. He thinks AI companies should be working with psychiatrists and mental health hospitals to study how people with different psychiatric conditions respond to chatbots, under controlled conditions, rather than waiting for more people to be harmed.
Ultimately, though, “it should have stayed in the lab for longer. We should have taken more time to understand what these systems are and how they can go wrong.
“But we live in a capitalist system, and these systems are expensive to run, so the profit motive took over.”
Photograph by The Washington Post / Getty Images
Artificial intelligence
Mental health
Google
OpenAI
Health
Young people
Technology
ChatGPT
Anthropic
