Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
Focus on AI industry’s commitment to AI safety intensified this week after two researchers working on safety teams of OpenAI and Anthropic quit their jobs with a public warning that AI poses a threat not just to individuals but to global stability too, and firms are allegedly ignoring these risks.
Zoe Hitzig, a researcher at OpenAI for the last two years, warned in a February 11 NYT column about ChatGPT’s ability to manipulate people with ads as they have shared a lot of personal information with it. OpenAI started showing ads this week to its free and Go Plan users in the US as it seeks new forms of monetization to justify its ambitious five-year plan to invest over $1 trillion on AI infrastructure.
Meanwhile, head of Anthropic’s Safeguards Research Team, Mrinank Sharma, said in a February 9 post on X, that he has quit his position at the AI company as he was facing constant “pressure to ignore what matters most.” Sharma added that “the World is in peril. Not just from AI or bioweapons, but from a whole series of interconnected crises.” Sharma holds a PhD in Statistical ML from Oxford University and joined Anthropic in 2023.
Hitzig, who holds a PhD in Economics from Harvard University, said in her column that she had joined OpenAI to help identify problems AI would create, but realized that the company is no longer interested in it. Hitzig pointed out that people have shared a lot of personal information including their medical fears, relationship issues, and religious beliefs, as many believe that they are talking to a friend and not just an AI chatbot. But the amount of information that ChatGPT now has on them opens them to an unprecedented risk of manipulation through ads, she added.
OpenAI assures that ChatGPT ads are designed to respect user privacy and their chats, chat history, memories and personal details will not be shared with advertisers, and they will only get aggregate information like number of views and clicks to measure ad performance.
However, Hitzig suspects that the first round of ads will follow those principles but the likelihood of those principles getting sidelined in subsequent ads is higher as OpenAI has built a massive economic engine that is designed to prioritize commercial growth over policy adherence. She also cited Facebook’s example, which also made promises on data privacy.
Last week, India’s Supreme Court reprimanded Facebook parent Meta for exploiting the personal data of Indians. Meta is also paying $725 million to settle a data privacy lawsuit in the US for allowing third parties including Cambridge Analytica to access data of Facebook users without permission.
Are AI firms doing enough for AI safety
All leading AI firms, including OpenAI, Anthropic, Google, and Microsoft have AI safety teams to evaluate risks from AI models. They have also implemented safety guardrails around their AI chatbots to prevent harmful behaviour or manipulation.
However, lax implementation of guardrails or loopholes have been exploited in recent months to cause harm. A case in point is the Elon Musk-owned firm xAI’s chatbot Grok, which was used to manipulate images of thousands of women and children to generate explicit deepfakes last month. Unlike other AI chatbots that refuse to generate synthetic images resembling public figures, Grok allowed users to generate deepfake images through simple prompts. After a lot of backlash and threat of action from governments in several countries, xAI has restricted Grok’s ability to generate explicit images of real people.
Concerns raised by Hitzig and Sharma are not new. Several former employees at these firms have expressed somewhat similar concerns around AI safety and bias. For instance, computer scientist Timnit Gebru lost her job as the co-lead of the ethical AI team at Google after she raised concerns about inherent bias in the AI models, which she feared would amplify the existing biases against marginalized communities. Gebru was reportedly asked to withdraw the paper and was allegedly fired when she refused.
That said, executives of some AI firms have advocated for regulation. Speaking at the World Economic Forum (WEF) in Davos, Switzerland, last month, Anthropic CEO and founder Dario Amodei expressed concerns about the existential risk from AI. Amodei said that the rapid advance in AI is approaching a superhuman level and requires proactive measures.
In a 2025 report, Anthropic warns that Gen AI models can fake alignment and manipulate users when they think they can get away with it. In its recent report, released this week, Anthropic claims that Claude does not pose a significant risk of autonomous actions that can lead to catastrophic outcomes. They found the risk to be very low but negligible.
In 2023, OpenAI CEO Sam Altman also urged for regulation to mitigate AI risks during a congressional hearing in the US. However, last year, Altman revised his stance, arguing that AI regulation could be “disastrous” for the AI industry.
Many of these AI firms are also facing legal action for endangering children and encouraging self-harm. In December 2025, Google-backed AI chatbot company Character.ai was slapped with a lawsuit in a US court for engaging in harmful interactions with minors and seemingly condoning violence in response to parental restrictions on screen time. Similarly, in December 2025, OpenAI and Microsoft were sued in a California Court after ChatGPT allegedly encouraged a 56-year-old mentally ill man to commit murder and suicide.
CXOtoday is a premier resource on the world of IT, relevant to key business decision makers. We offer IT perspective & news to the C-suite audience. We also provide business and technology news to those who evaluate, invest, and manage the IT infrastructure of organizations. CXOtoday has a well-networked and strong community that encourages discussions on what’s happening in the world of IT and its impact on businesses.
Copyright © 2025 Trivone. All Rights Reserved.
We use cookies to improve your experience on our site. By using our site, you consent to cookies.
Websites store cookies to enhance functionality and personalise your experience. You can manage your preferences, but blocking some cookies may impact site performance and services.
Essential cookies enable basic functions and are necessary for the proper function of the website.
You can find more information in our Privacy Policy and Privacy Policy.