Study exposes enterprise's data risk from AI chatbots – TechInformed

Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
Study exposes data risks for businesses using leading AI chatbots
July 1, 2025
New research from data privacy firm Incogni has raised fresh concerns about how major generative AI chatbots handle personal and business data.
The study warns that large language models developed by the likes of Meta, Google and Microsoft collect sensitive information and share it with third parties — often without adequate transparency or meaningful user control.
Incogni’s analysis of leading AI platforms, including Google Gemini, Meta AI, DeepSeek, Pi.ai and Microsoft Copilot looked across 11 subcategories in three key areas: how user data is utilised in model training, the transparency of each platform’s privacy practices, and the scope of data collection and third-party sharing.
The results show that these tools gather a wide range of data. This includes names, email addresses, phone numbers, precise location information and, in some cases, physical addresses. Few of the platforms provide clear mechanisms for businesses or individuals to opt out of having their data used for AI training.
While regulations such as GDPR grant individuals the right to request data erasure, it’s still unclear how to practically remove the information from a machine learning model.
As a result, many companies are not currently obligated, or technically able, to remove such data after the fact.
Contact details or proprietary business details may become embedded in the model’s training data, potentially without the user’s explicit knowledge or consent.
The study highlights the risks for enterprises. Employees using generative AI tools to draft reports, emails or code may inadvertently expose proprietary information, which could end up embedded in AI training datasets.
This introduces significant compliance, confidentiality and competitive risks, particularly for organisations handling sensitive client data or intellectual property.
Key findings from the report include Meta.ai sharing names and contact details with external partners, Claude disclosing email addresses and app interactions to third parties, and Grok (xAI) potentially sharing user-uploaded images.
Microsoft’s privacy policy suggests user prompts may be shared with advertising partners.
Nike, H&M, The North Face among apps sharing sensitive data, study finds
While OpenAI’s ChatGPT was noted in the report for clearer privacy policies, experts stress that all generative AI platforms require cautious and informed use when it comes to sensitive data.
“Many businesses remain unaware that routine use of AI tools could see internal data reused or shared in ways that undermine confidentiality and compliance obligations,” said Darius Belejevas, head of Incogni. “The lack of transparency and opt-out options creates real exposure for firms.”
The report calls for stronger safeguards, clearer privacy disclosures, and practical ways for organisations to prevent their data from being retained or reused by AI platforms.
For businesses adopting AI at scale, robust internal policies and supplier due diligence are now critical to managing data security and regulatory risk.
by James Pearce
by Nicole Deslandes
by Nicole Deslandes
Insights
Uncategorized
Subscribe to our Editor’s weekly newsletter
Stay informed, let’s connect:
©2025 powered by IResearch Services. All rights reserved.
SVP Global Marketing, TechInformed
With over 30 years of global marketing experience working with industry leaders like IBM, Intel, Apple, and Microsoft, Amy has a deep knowledge of the enterprise tech and business decision maker mindset. She takes a strategic approach to helping companies define their most compelling marketing stories to address critical obstacles in the buyer – seller journey.
Editor, TechInformed
As founding editor of TechInformed in 2021, James has defined the in-depth reporting style that explores technology innovation and disruption in action. A global tech journalist for over a decade with publications including Euromoney and IBC, James understands the content that engages tech decision makers and supports them in navigating the fast-moving and complex world of enterprise tech.
This website protects your privacy by adhering to the European Union General Data Protection Regulation (GDPR). Please tell us which of the following data-processing activities you agree to. We will not use your data for any other purposes. Learn More
This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.
Strictly Necessary Cookie should be enabled at all times so that we can save your preferences for cookie settings.
If you disable this cookie, we will not be able to save your preferences. This means that every time you visit this website you will need to enable or disable cookies again.
This website uses Google Analytics to collect anonymous information such as the number of visitors to the site, and the most popular pages.
Keeping this cookie enabled helps us to improve our website.
Please enable Strictly Necessary Cookies first so that we can save your preferences!