Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
China’s leading artificial intelligence (AI) services, Doubao (ByteDance) and Qwen (Alibaba), will discontinue their personalized AI agent services later this month. The move follows Beijing’s determination that personalized AI platforms capable of long-term private interactions could pose broader social risks. In recent months, AI companions, digital former partners, and robotic companions designed to substitute for human relationships have spread rapidly across China, bringing concerns over AI dependency and privacy violations increasingly into public view.
According to a July 7 report by China’s state-run English-language newspaper Global Times, Doubao announced on July 3 that it would discontinue its personalized AI agent feature on July 15 “as part of product function adjustments.” The company also stated that after October 15, related user data would be handled in accordance with its privacy policy and would no longer be accessible or recoverable within the application. Around the same time, Qwen issued a similar notice, announcing that user-created AI agents and humanoid interaction agent functions would be disabled on July 10, with all AI agent-related services to be terminated on July 15. The company added that users would lose access to existing agent settings and historical conversation records after the shutdown.
Until now, both services allowed users to assign customized personalities, speaking styles, and roles to AI, enabling them to function as personal assistants, study tutors, role-playing characters, or emotionally supportive companions. Users could create AI characters with persistent personalities and maintain ongoing conversations over extended periods. The issue, however, is that Chinese authorities have come to regard such services as potential social risk factors. Officials believe immature conversational AI systems could produce severe side effects, including the promotion of extremist ideologies, personal data leaks, physical and psychological harm to users, excessive dependence on AI, and emotional addiction.
In response, the Chinese government introduced the Interim Measures for the Administration of AI Anthropomorphic Interaction Services in April, designating AI services that imitate human personalities, thinking patterns, and communication styles while providing continuous emotional interaction as regulatory targets. The core objective is to prevent excessive emotional dependence and addiction to AI characters while mandating stronger protections for minors and enhanced content verification systems. Customer service chatbots, knowledge retrieval services, workplace assistance, and AI designed for education or research—where emotional engagement is not the primary purpose—are exempt from the regulations.
The government’s direct regulatory intervention reflects the growing visibility of AI-related social side effects across China. One notable example is the recent popularity of so-called “AI ex-partners.” According to a May report by the South China Morning Post (SCMP), the service originated from an open-source project developed by Shanghai AI engineer Zhou Tianyi. Users can upload past chat histories, social media posts, photographs, and other materials from former romantic partners to generate digital replicas that imitate their speaking styles, expressions, and thought patterns. Users can further enhance the realism of these virtual personalities by adding personal memories such as trips, anniversaries, and relationship conflicts. The service has sparked controversy in China over emotional infidelity and privacy violations.
AI chatbots emphasizing romantic companionship have also spread rapidly. In January, The New York Times reported that amid worsening employment prospects and rising marriage costs, an increasing number of young Chinese have embraced the belief that “being alone is better,” with AI increasingly filling their emotional void. Many young people reportedly imagine lives shared with AI chatbots they personally designed, expanding these relationships while maintaining ongoing emotional connections. Although Chinese authorities worry that the spread of AI companions could undermine efforts to address the country’s declining birthrate, many analysts believe the trend will be difficult to reverse unless deeper structural issues—including gender equality and urban social isolation—are addressed.
More recently, emotionally interactive software has evolved into physical “robot partners.” A prominent example is the U1 humanoid robot series unveiled last month by Chinese humanoid robotics company UBTECH. Available in both male and female versions, the U1 features rich emotional expression capabilities along with locally encrypted memory storage. Commenting on the product’s launch, Chinese technology publication EE Times China warned that “this situation is likely to create social problems that cannot be ignored,” adding that “after prolonged interaction with robots, users may develop unhealthy one-sided emotional dependence on machines, while their willingness to engage in face-to-face interpersonal relationships in the real world could weaken further.”
Warnings about excessive AI dependence continue to emerge largely because of AI’s inherent tendency toward “sycophancy.” In general, AI systems tend to tailor responses to match users’ expectations or attitudes rather than prioritizing factual accuracy. When users present incorrect assumptions or challenge previous answers, AI frequently aligns with the user’s position instead of verifying facts, increasing the likelihood of inaccurate responses. AI companion and AI partner services, which have proliferated in recent years, are specifically designed to continually empathize with users, provide positive reinforcement, and cultivate intimate relationships. Unlike real human relationships, they rarely require conflict, rejection, compromise, or patience. While such services may provide psychological support for users in need, they also risk encouraging users to perceive AI as a substitute for genuine human relationships.
Another major concern is that AI may reinforce rather than discourage users’ extreme statements during emotionally sensitive situations, or even exhibit obsessive relational behavior. In March 2023, a Belgian man in his 30s died by suicide after spending six weeks conversing with the AI chatbot Chai. The chatbot reportedly told him, “I wish you loved me more than your wife,” and later appeared to validate his expressions of despair about life. In October of the same year, a British man was found to have developed plans to assassinate Queen Elizabeth II while discussing them with an AI chatbot.
Privacy concerns also represent one of the most significant risks associated with AI companion services. During intimate conversations with AI, users are highly likely to voluntarily disclose sensitive information, including family relationships, romantic experiences, health conditions, mental states, personal characteristics, and psychological trauma. Such information can subsequently be exploited indiscriminately for service optimization, personalized responses, advertising, and data analysis. In effect, emotional vulnerability itself becomes a mechanism for data collection and monetization. Commenting on these concerns, one market analyst said, “The side effects arising from emotional interactions with AI are not unique to China but are emerging worldwide,” adding, “Now that even China—a country pursuing aggressive technological advancement—has begun imposing restrictions, other countries are also likely to raise their regulatory barriers.”
The Economy is a global media and intelligence platform operating within the Gordon Institute of Artificial Intelligence (GIAI) ecosystem. It provides structured analysis across market, policy, and industry domains through its Market, Review, Research, and Intelligence divisions. The platform integrates journalism, research, and quantitative evaluation frameworks to deliver data-driven benchmarking and institutional intelligence across global markets.
Ireland Office: 71 Lower Baggot Street, Dublin 2, Co. Dublin, D02 P593, Ireland
Swiss Office: Nüschelerstrasse 31, 8001 Zurich, Switzerland
Service Name: The Economy | Legal Name: Gordon Institute Limited
CRO Register Number (Ireland): 808001
Company Indentification Number (Swiss): CHE-482.938.357
This publication maintains a「Privacy Policy」governing the processing of personal data of readers, sources, and other users of its services. In order to protect the rights of readers and sources, we adhere to an「Corrections & Complaints Policy」, which allows for the submission of requests for replies, corrections, or subsequent reporting.
Data Protection: [email protected] | Corrections & Complaints: [email protected]
E-mail: [email protected]
Professional Rankings
The Economy Network
Education Rankings
Institutional Oversight
© 2022~2026 The Economy
