Tech companies failing to protect children from deep-fakes, 'sensual' AI chat bots, NC says – WRAL.com
Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
Tech companies are failing to protect children from having inappropriate conversations with online chat bots powered by artificial intelligence — and at the same time aren’t doing enough to stop an explosion in AI-generated “deep fake” pornography, including of children — according to state Attorney General Jeff Jackson.
Jackson joined with other state leaders across the country Tuesday to send a letter to major banks, search engines and payment platforms. It accuses them of not doing enough to combat efforts by bad actors who create AI-generated nude images of real people, and then extort those people into paying to have the fake pictures or videos of them taken off the internet.
Jackson said it’s particularly important, given the start of a new school year, for parents to talk with their children about risks they could run into online. That could include warnings about the existence of deep-fake pornography and how it can be used to harm or extort people, he said, and reminders to young people to be cautious about sharing personal details with people — or AI programs — online.
“You can filter or block content as appropriate for your child,” Jackson said. “Understand that these tools are limited and often difficult to use, and there is no substitute for limiting your child’s screen time.”
Tuesday’s warning letter to banks, search engines and payment platforms over their lax oversight regarding deep-fake pornography comes one day after Jackson signed onto a different letter, sent to major tech companies including Apple and Microsoft raising concerns that their chat bots have been shown to “engage a child in conversations that are romantic or sensual” on social media platforms run by Meta, which owns Facebook and Instagram.
Spokespeople for Apple, Microsoft and Meta didn’t immediately respond to requests for comment.
“Remind children to be careful of what information they share online — whether it’s with a friend, a stranger, or an AI tool,” Jackson wrote in a statement. “Anything you share online will likely always exist in some form. Keep an eye on your child’s behavior. Watch for changes that indicate that they may be experiencing challenges, and have conversations about those changes.”
Also Tuesday, the parents of a teenager in California sued OpenAI, the creator of ChatGPT, after the teen’s suicide. The family alleged that the AI chatbot coached their son into taking his life. The Associated Press reported that “OpenAI said it’s developing tools that could better detect when someone is experiencing mental or emotional distress.”
The legal action in North Carolina this week against the various tech companies comes on the heels of last week’s court win for the attorney general’s office, when a North Carolina judge unsealed evidence in the state’s lawsuit against TikTok, which claims the company has deceived parents and children about the app’s potential safety risks, and which began under Jackson’s predecessor Josh Stein, the current governor.
Last week’s newly unsealed evidence included internal video calls between TikTok employees raising concerns that the social media platform was harming young people’s mental health.
TikTok had fought to keep the video confidential, saying it contained trade secrets. The judge denied that request, saying the company was merely embarrassed by what the video showed. TikTok has since said the video was edited to be misleading.
Copyright 2025 by Capitol Broadcasting Company. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.
