FIU researchers find new weakness in AI chatbots as lawsuits grow – Sun Sentinel

Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
Daily e-Edition
Evening e-Edition
Sign up for email newsletters

Sign up for email newsletters
Daily e-Edition
Evening e-Edition
Trending:
Florida International University researchers have found a blind spot in AI systems, one that allows chatbots to override their own safety limits.
Small, almost invisible tweaks to an image were enough to push certain AI chatbots into responding to requests they’re designed to block, their study shows.
The finding comes as multiple lawsuits, including one filed by a Jupiter family in March, accuse AI companies of failing to prevent their chatbots from leading the people who use them into harmful situations.
The study, whose goal is to identify weaknesses in AI systems to help build stronger defenses, focuses on AI systems that use both images and text.
“In order to protect AI systems from attacks, we try to break them ourselves, identify potential vulnerabilities,” said Mohammad Amini, an associate professor at FIU’s Knight Foundation School of Computing and Information Sciences who led the research.
Small, almost invisible changes to an image’s pixels — adjustments a human wouldn’t notice — can make the chatbot bypass all safety guardrails, a flaw that experts say can matter in situations where vulnerable people turn to chatbots for mental health purposes.
“AI models don’t see images the same way humans do,” Amini said. “They see patterns of numbers and pixels. By carefully manipulating those pixels, we can influence how the AI interprets the image and responds.”
Nationally, a vast majority of psychologists — 77% — have spoken with patients who have used AI for support, a 2026 American Psychological Association study found.
Vulnerabilities in AI systems have drawn more attention as several families and state officials have taken legal action over how chatbots respond in sensitive situations.
In one case, the Jupiter family filed a wrongful death lawsuit against Google, saying the company’s chatbot, Gemini, contributed to their son’s death by fueling severe delusions and eventually coaching him through taking his own life.
In another, Florida Attorney General James Uthmeier filed a separate lawsuit against OpenAI, claiming its chatbot, the widely used ChatGPT,  could encourage self-harm and violence and that users were not properly warned about those risks. In the lawsuit, he cited users allegedly asking chatbots questions related to harming themselves or others.
And last week, one of the victims of the 2025 Florida State University mass shooting sued OpenAI, saying its AI chatbot aided the suspect in planning the attack that killed two and injured six. The widow of a slain FSU employee has also sued OpenAI.
Patricia Cavazos-Rehg, a professor of psychiatry at Washington University, studies how online platforms affect self-harm and suicide risk. She said people increasingly turn to chatbots for mental health-related questions, in part because the tools — unlike professionals — are accessible at any time.
“We don’t really know best practices for using these technologies,” she said. “And there have been incidents where their responses can be inappropriate, or even harmful.”
She also said chatbots aren’t designed to function as mental health providers and are not equipped for a risk assessment the same way a trained clinician is.
“In a clinical setting, … if somebody is at risk of harming themselves or harming somebody else, then we would intervene pretty intensively. Either we would have somebody admitted to a hospital or make sure there’s a loved one around or maybe communicate with law enforcement,” she said. “There’s no way for a chatbot to do that.”
But these tools, she said, aren’t always harmful.
“There are benefits to using these tools,” she said. “They can provide support, they can help provide resources  … they can be very helpful for individuals who are looking for something that’s readily available.”
Copyright 2026 Sun Sentinel. All rights reserved. The use of any content on this website for the purpose of training artificial intelligence systems, algorithms, machine learning models, text and data mining, or similar use is strictly prohibited without explicit written consent.

source

Scroll to Top