Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
Home
What's the word?
Test your knowledge with our new weekday mini crossword
By , , and
Oct. 26, 2026
Yang is an assistant professor in the Bowers College of Computing and Information Science at Cornell University, where she directs the DesignAI Group. Cooper is a postdoctoral researcher in the DesignAI Lab at Cornell. Emara is a graduate student at the Cornell Brooks School of Public Policy.
OpenAI recently announced that it is tracking signs of user distress across sessions, rather than only within them, and using that information to inform model responses. The announcement follows a wave of new safeguards for mental health and well-being conversations with artificial intelligence at OpenAI and elsewhere, such as parental controls for teens, usage limits and break reminders, and crisis detection and referral. Policymakers have also moved to address associated risks, with a dozen states now requiring companion chatbot operators to disclose to users that their tools are not human, and to institute crisis referral procedures.
This attention, within AI companies and legislatures, is important and urgent. But it is directed almost entirely at acute harm from interacting with AI, rather than the cumulative harm that develops over time, such as the gradual replacement of one’s connections with friends and family by AI interaction.
Advertisement
Acute harms, such as psychosis or suicide reportedly linked to specific chatbot interactions, are easier to identify and respond to than harms that build over months and years. A chatbot could be safe in every single conversation with a user and still, across months of use, displace the human relationships that keep people healthy.
Early studies suggest that the use of AI chatbots for companionship or emotional support is concentrated among a subset of heavy users, and that heavier use is associated with reduced socialization and increased loneliness. Although this evidence is preliminary and debated, the findings warrant further measurement.
AI companies collect aggregate user data that could help answer these questions, but broader participation and greater access to this data is necessary. If AI use is not displacing human connection at scale, data and robust measurement should show that too. If we don’t start measuring today, and the harm is real, the window in which we might have acted could be closed by the time it becomes undeniable.
Advertisement
AI companies have predominantly focused on per-message and per-session interventions, such as break reminders and usage limits. Across the public commitments of major companies (in model specifications or “constitutions,” usage policies, and safety blog posts), safeguards consistently operate on individual messages or sessions. Crisis detection and referral, too, largely operated at the same level until OpenAI’s announcement that it is tracking distress across sessions. This change demonstrates that companies have the capability to track context about users, but tracking user distress is still focused on detecting acute harms.
Tracking displacement is different, as a user gradually pulling back from their human relationships may show few, if any, signs of distress. Indeed, a user may feel that each individual chat session itself is helpful. Researchers and policymakers like us don’t have a shared understanding of how to measure whether such displacement is occurring across sessions, nor do we have the necessary visibility into the usage data to do so.
The lack of attention to cumulative harms is also important as, in other domains, industry actors have repurposed product-level safeguards as legal or rhetorical defenses when accused of contributing to cumulative harm. The tobacco industry, for example, used warning labels as the basis of assumption-of-risk defenses that stopped individual liability for decades. More recently, Meta has highlighted parental supervision tools and “Take a Break” notifications in congressional testimony, even as it has denied a causal link between social media use and harms to adolescent mental health.
We do not accuse AI companies of bad faith in instituting product-level safeguards, nor do we suggest companies must choose between addressing acute harms and preparing for cumulative ones. However, the historical cases suggest that if accountability for cumulative harm arrives, product-level safety features deployed in isolation can be repurposed as part of a defense to such claims. Per-chat reminders to users that they are “not talking to a human” could feasibly become such features.
Advertisement
What eventually made tobacco’s cumulative harm actionable was that states showed their Medicaid budgets absorbing the downstream costs of industry products. State attorneys general did not have to prove that a product harmed a given person once they could measure and account for the cost to public systems. That accounting already existed when the legal and political window opened.
There exists no equivalent measurement infrastructure to understand whether AI use displaces human connection. AI companies hold usage data closely, while public systems, from health to education, know what social isolation costs them. There is scant connection between the two. Frontier labs are already analyzing aggregate data on how people use these tools, in privacy-preserving ways, and have recently provided access to some third parties to analyze how models respond to mental health crises. However, this has not yet been applied to cumulative harm.
Three groups could start to change this picture. Researchers in academia and civil society should develop the measurement approach and instruments to distinguish AI chatbot use from other contributors to social isolation. Public health, education, and other agencies can measure and share what social isolation already costs them, in areas such as school counselor time, primary care utilization, or demand on social prescription programs that are expanding in the U.S. Finally, the companion chatbot bills in front of state legislatures are an opportunity for legislators to encourage greater access for independent third parties to conduct evaluations related to social isolation.
These actions don’t require settling causation between AI chatbot use and social isolation. However, they do require putting the measurement infrastructure in place before harm becomes undeniable. Many of us have watched the debate about links between social media use and adolescent mental health run for more than a decade without consensus, thanks in part to the lack of necessary measurement infrastructure to inform policymakers.
Advertisement
If the policy window opens to address the effects of AI use on people’s relationships in years to come, policymakers will ask what we know. We already know the value of social connection to our health. What we don’t know is whether people are disconnecting from one another, gradually, over months of interacting with chatbots. By setting up measurement infrastructure we can start to understand, while the answer can still make a difference.
Qian Yang is an assistant professor in the Bowers College of Computing and Information Science at Cornell University, where she directs the DesignAI Group. Ned Cooper is a postdoctoral researcher in the DesignAI Lab at Cornell, where he studies product design and safety guardrails for AI-powered mental health and well-being tools. Ibrahim Emara is a graduate student at the Cornell Brooks School of Public Policy and studies applied projects around responsible AI.
Have an opinion on this essay? Submit a letter to the editor.
Qian Yang
Ned Cooper
Ibrahim Emara
The smartest thinkers in life sciences on what's happening — and what's to come
Your data will be processed in accordance with our Privacy Policy and Terms of Service. You may opt out of receiving STAT communications at any time.
Torie Bosch
Advertisement
, , and
By Tara Bannow
By Allison DeAngelis
By
By Jason Mast
By Katie Palmer
Share options
X
Bluesky
LinkedIn
Facebook
Email
Doximity
Copy link
Reprints
Reporting from the frontiers of health and medicine
Company
Account
More
