Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
Artificial intelligence chatbots make two kinds of errors.
Complete the form to unlock this article and enjoy unlimited free access to all PYMNTS content — no additional logins required.
yesSubscribe to our daily newsletter, PYMNTS Today.
By completing this form, you agree to receive marketing communications from PYMNTS and to the sharing of your information with our sponsor, if applicable, in accordance with our Privacy Policy and Terms and Conditions.
A hallucination is a false output, like a made-up citation, a wrong number or a name that doesn’t exist.
A confabulation is different. It happens when a system doesn’t know the answer and fills the gap with something that sounds like it fits, stated with the confidence of a real policy or a real customer service response.
A confabulated policy, delivered through a company’s own support channel, looks like the real thing. To the customer reading it and to the court reviewing it afterward, it carries the same authority.
The legal exposure crystallized in February 2024, when the British Columbia Civil Resolution Tribunal ruled Air Canada liable for a chatbot’s fabricated bereavement fare policy, in the case Moffatt v. Air Canada, 2024 BCCRT 149.
Jake Moffatt consulted Air Canada’s chatbot after his grandmother died and was told he could buy a full-fare ticket and claim a retroactive bereavement discount within 90 days. However, no such retroactive policy existed.
When Air Canada denied Moffatt’s claim, it said the chatbot was responsible for its own statements, the American Bar Association reported at the time. The tribunal rejected that argument directly.
“In effect, Air Canada suggests the chatbot is a separate legal entity that is responsible for its own actions,” tribunal member Christopher Rivers wrote in the decision. “This is a remarkable submission.”
The tribunal ordered the airline to pay Moffatt 812.02 Canadian dollars (about $570) in damages and fees.
The ruling established that companies are liable for statements their AI systems make on their behalf, regardless of whether those statements were intentional or the product of a model filling a knowledge gap.
The next test arrived in April 2025.
An AI support bot at AI coding startup Cursor told a developer that the company had a new policy limiting each subscription to a single device, Ars Technica reported at the time. The policy did not exist. The bot, named Sam, had invented it.
Cursor Co-Founder Michael Truell confirmed the fabrication on Reddit.
“Hey! We have no such policy,” Truell said. “You’re of course free to use Cursor on multiple machines. Unfortunately, this is an incorrect response from a front-line AI support bot.”
Before the correction reached users, the invented policy had circulated on Hacker News and Reddit.
In May 2025, Lloyd’s of London launched an insurance product specifically covering AI hallucination-related losses. The policies, offered through a startup called Armilla, cover court claims against a business if it is sued by a customer or third party harmed by an underperforming AI product.
FINRA’s 2026 Annual Regulatory Oversight Report flagged hallucinations as a specific compliance concern for broker-dealers, warning firms to develop procedures for AI agents that may act beyond the user’s intended scope. The regulator defined hallucinations as instances where a model generates information that is inaccurate or misleading yet presents it as factual.
Scaled Cognition raised $100 million in June to build enterprise-grade hallucination controls. The investment signals that the cost of confabulation, once treated as a product quality problem, is now being priced as a financial risk.
The pattern the Cursor and Air Canada cases share is not a failure of AI to understand the question. It is a failure of the system to recognize that it does not know the answer, and to say so. When the invented answer concerns a company’s pricing, policy or refund terms, courts, regulators and insurers now agree: The company owns what its model said.
For all PYMNTS AI coverage, subscribe to the daily AI Newsletter.
Courts to Companies: You Own What Your Chatbot Says
Plaid Talks With Banks About Potential IPO
WhatsApp Steps Up Fight Against Account Impersonation
JPMorgan AI Chief Teresa Heitsenrether Plans Year-End Retirement
Get PYMNTS Today, AI, B2B and more.
We’re always on the lookout for opportunities to partner with innovators and disruptors.