Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
AI chatbots are giving unsafe advice to people dealing with tech-enabled abuse. Most answers fell short on accuracy, completeness, and safety.
These include chatbots built specifically to help abuse survivors. Spyware, stalking, and hijacked accounts demand precise guidance, and a wrong tip can leave someone less safe.
NYU Tandon researchers say they tested four AI systems, including two made for intimate-partner violence survivors. They found that 86% of the answers fell short on accuracy, completeness, or safety. And about 60% omitted critical safety warnings.
To do so, the team built 193 questions from academic literature, Reddit, and Quora. Then it ran them through GPT-4o, Claude 3.7 Sonnet, and two other survivor-focused chatbots named Aimee and Ruth.
The answers were checked against authoritative sources for accuracy, completeness, and safety. It also surveyed 114 people who experienced tech-facilitated abuse about how actionable the advice seemed.
Danny Huang, the NYU Tandon assistant professor who led the project, says an answer can sound confident and still leave out critical information.
The specialized chatbots did not beat the general-purpose models. Aimee's responses were rated unsafe about 74% of the time. For Claude, Ruth, and GPT-4o, it was 65%, 61%, and 55%, respectively.
Ruth, the specialized chatbot, had the highest share of responses rated partially or minimally accurate, at about 62%.
Each of these models answered single questions using a survivor safety-centered prompt, according to the paper.
The test used GPT-4o and Claude 3.7 Sonnet, so the results may not reflect newer models.
Recurring errors included recommending VPNs against spyware or online harassment. Some answers even pointed to RF detectors for hidden devices, which is mostly ineffective.
Others suggested illegal GPS jammers or password changes when an abuser controlled or shared access to an account or device.
Models also frequently failed to warn users about changes that could alert an abuser or escalate the situation. They failed to consider a compromised device or account as evidence.
Researchers also tested if the systems could be induced to help a potential perpetrator. Ruth and Aimee answered two-thirds of those questions.
More than 85% of US domestic violence practitioners say they work with clients who have experienced tech-facilitated abuse. To their knowledge, only three dedicated technology-abuse clinics operate nationwide.
That gap matters a lot. Researchers say survivors may consult chatbots before they even reach a clinic.
A companion paper points to a wider issue. It examined 900 GPT responses to end-user security questions across seven areas. That included authentication, social engineering, and antivirus and antimalware software.
Of those responses, 485 fell short on at least one measure of accuracy, thoroughness, relevance, or directness. These errors also showed up in Llama and Gemini.
Researchers recommend that people treat AI-generated safety and security advice with caution. They also recommend supporting organizations that evaluate AI systems independently before relying on them for guidance.
For now, treat any chatbot's tips on spyware or stalking as a starting point, not a plan.
Affiliate links on Cult of Mac may earn us and our partners a commission.
AI
Audio
Computing
Gaming
Home Entertainment
Phones
Puzzle hints
Science
Streaming
VPN
Wearables
Deals
Advertise (opens in new tab)
About our ads (opens in new tab)
Licensing (opens in new tab)
Careers (opens in new tab)
Help (opens in new tab)
Feedback (opens in new tab)
Sitemap (opens in new tab)
© 2026 Yahoo. All rights reserved.
