Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
AI chatbots are not subject to federal HIPAA laws that protect your personal medical information. (Illustration by Galina Nelyubova for Unsplash+)
Sept. 9, 2026 — As AI chatbots become more integrated into our daily lives, some people are becoming more comfortable with asking Claude medical questions and divulging their personal health information to ChatGPT.
That’s a big mistake. You should never give your personal medical information to a chatbot. Here’s why.
The U.S. federal law that guards the safety and privacy of your personal medical information is HIPAA, the Health Insurance Portability and Accountability Act of 1996.
The HIPAA Privacy Rule keeps your personal health information private and protected. Organizations subject to HIPAA rules, called “covered entities,” include healthcare providers, insurance plans, and their business associates (usually third-party billing and claims processing companies).
AI chatbots like ChatGPT, Claude, Google Gemini and Microsoft Copilot are not covered entities. They are not subject to HIPAA laws.
That means no matter how many assurances the chatbot or its developer may offer, there are no rules that prohibit the AI operator from revealing, selling, or misusing any personal medical information you enter as a prompt.
Resource: Is ChatGPT HIPAA Compliant? (The HIPAA Journal)
Some medical offices have incorporated AI into their digital record systems. If they have, they most likely have taken care to customize specific AI tools that remain within a larger HIPAA-compliant IT infrastructure.
They’re not just uploading your medical records into a public chatbot.
Resource: Why Doctors Using ChatGPT Are Unknowingly Violating HIPAA (USC Leonard D. Schaeffer Institute)
Chatbots remember everything. AI developers may tout this as a beneficial feature. Chatbots like Claude and ChatGPT remember what you have told them and use that information to shape future responses. To personalize things.
The downside is that a chatbot’s “forever memory” means it is keeping a data file on you. Over time, the more data that accumulates in that file the more valuable it becomes—not just to the AI operator but on the open market.
Tech companies are widely known to monetize data any way they can.
In July 2026, the Federal Trade Commission sued Hims & Hers Health Inc., the online pharmacy, for allegedly sharing their customers’ private health information with Meta and Snap. The two social media companies wanted the data to better target those patients with ads.
When the ancestry company 23 and Me stood on the verge of bankruptcy in 2024, the DNA data of millions of customers went up for sale. 23 and Me wasn’t bound by HIPAA, and the fine print in the company’s privacy policy stated that in the event of a merger or acquisition, customer information would be treated as a salable asset.
Data breaches, phishing attacks, and information theft happen all the time. Medical companies and their HIPAA-covered business associates are highly incentivized to keep their data private and secure. Chatbot developers, not so much.
AI chatbots are not subject to federal HIPAA laws that protect your personal medical information. What could go wrong? Plenty.
The problem of AI hallucination has been with us since the earliest days of ChatGPT—and computer scientists are still confounded by the issue. Hallucinations happen when an AI system generates information that appears plausible but is factually incorrect.
Chatbots are designed to respond in a friendly and highly confident tone, making it difficult to tell which answers are accurate and which are complete fiction.
ChatGPT is wrong so often that OpenAI has had to add a permanent disclaimer to its home page: “ChatGPT is AI and can make mistakes.”
As one researcher has noted, “In healthcare, where accuracy and reliability are paramount, AI hallucinations can have severe consequences.”
Resource: Generative AI Hallucinations in Healthcare (Springer Nature)
It can be tempting to use a chatbot as a free, always-ready therapist.
That’s not a good idea.
Chatbots are not designed to respond with thoughtful, medically informed answers. They are designed with one goal in mind: To keep you engaged as long as possible.
For the companies operating the chatbot, engagement time is money. The most recent chatbots have been designed as “multi-turn” machines, purposely built to encourage the human user to continue the back-and-forth interactions that produce more screen time (to sell more ads), more personal data about the user (to better target those ads), and more human responses for the AI model to learn from.
That’s why sycophancy is such a problem. Chatbots are programmed to be overly agreeable. They are flattery machines. They will tell you you’re brilliant, that your ideas are groundbreaking and visionary. People with no history of mental illness can spiral into delusions brought on by a chatbot. They will literally cheer you on to your death.
Therapy chatbots are so ill-advised that a number of states have outlawed them. Illinois, Colorado, Maine, Rhode Island, Tennessee, and Vermont have passed laws prohibiting any person or company from offering therapy services based on artificial intelligence.
Resource: Experts Caution Against Using AI Chatbots for Emotional Support (Columbia University)
AI chatbots are not subject to federal HIPAA laws that protect your personal medical information. What could go wrong? Plenty.
Seven states restricted the use of AI in medical procedure authorizations, and five others prohibited AI chatbots from replacing licensed mental health therapists. Here’s a look at everything that’s been approved in 2026 so far.
A coalition of civil society groups has filed a complaint asserting that therapy chatbots produced by Character.AI and Meta AI Studio are practicing medicine without a license. They are asking state attorneys general and the FTC to investigate.
Transparency Coalition
HOME
LEARN
NEWS
ABOUT
KEY TERMS
TERMS OF USE
PRIVACY POLICY
Follow Us!
Sign up with your email address to receive news and updates.
© 2026 Transparency Coalition.ai. All Rights Reserved