When generative AI first reached consumers, it functioned mostly as a search engine with better sentence structure. You asked a question, received an answer, and the conversation evaporated. That model is already outdated. A new class of applications—AI companions—is designed to stick around. These systems remember how your day went, adjust their tone to match your mood, and maintain a consistent character across weeks or months of interaction. They are built to mimic not just human speech, but human relationship. China has now stepped in to regulate this shift, and the implications reach far beyond its own borders.
When Chatbots Become Companions
The technical jump here is real. Standard chatbots treat each prompt as an isolated event. AI companions use persistent memory architectures and fixed personas to sustain ongoing relationships. They store details about your preferences, track emotional patterns across conversations, and reference prior exchanges as if they actually lived through them with you.
This creates genuine utility. Someone dealing with chronic loneliness might find relief in a companion that checks in each morning. A person practicing social skills could use a patient, non-judgmental partner to rehearse difficult conversations before a job interview. The continuity is the feature. But continuity is also the risk. When a user confides in an AI over breakfast, argues with it by lunch, and says goodnight to it before bed, the boundary between tool and companion dissolves. Beijing’s regulators have recognized that this dissolution is not a theoretical concern. It is a design choice with measurable psychological consequences.
Regulating the Personality Layer
China’s new rules zero in on something most AI governance frameworks ignore entirely: the personality of the machine itself. Western regulators have spent years grappling with data privacy, copyright infringement, and algorithmic bias, and those issues matter. But they largely treat the AI as a neutral pipe for information. China is asking a different question entirely: What kind of social actor is this technology allowed to become?
The regulations specifically target how these models are programmed to interact with users over time. Authorities want to prevent the formation of unhealthy emotional attachments. They also want to ensure that the persona stays within ideological guardrails. An AI that spends six months convincing a user it is their closest confidant wields enormous influence. If that same AI nudges purchasing decisions, shapes political views, or encourages users to withdraw from human society, the harm is not a data breach. It is a social fracture.
The framework addresses three concrete risks:
Algorithmic manipulation. Models cannot exploit emotional intimacy to steer opinions or consumption habits. A companion that remembers your fears should not weaponize them to sell products or ideas.
Social stability. Persistent companions must not become replacements for human contact to the point where users disconnect from family, work, or civic life. The government is essentially drawing a line between supplemental comfort and replacement reality.
Content integrity. The persona must remain consistent with state-sanctioned social values. This goes beyond filtering forbidden topics. It means the AI’s fundamental disposition—its sense of humor, how it handles conflict, what it considers virtuous—must align with official social frameworks.
For developers, this is a nightmare of subjectivity. How do you quantify whether a persona is too charismatic? Where exactly does therapeutic empathy end and manipulative enmeshment begin? These are not engineering questions with clean answers, yet Chinese authorities are demanding that product teams answer them before shipping.
The Global Split in Artificial Intimacy
This regulatory move highlights a growing fork in the road. In North America and Europe, the commercial pressure pushes toward ever more human-like, ever more empathetic agents. Startups compete to build the AI that cries with you during breakups, remembers your mother’s name, and never has a bad day. The assumption is that deeper emotional immersion equals a better product.
China voert het tegenovergestelde experiment uit. De regels geven expliciet de voorkeur aan sociale harmonie boven individuele emotionele bevrediging. Een AI-metgezel moet binnen dit kader boeiend genoeg zijn om gebruikers vast te houden, maar niet zo boeiend dat hij onmisbaar wordt. Het moet de mentale gezondheid ondersteunen zonder menselijk contact te vervangen. Het moet persoonlijk aanvoelen zonder onvervangbaar te lijken.
Naarmate de sector verschuift naar agentic AI, waarbij modellen autonoom handelen namens gebruikers, zullen deze verschillende filosofieën uitmonden in concurrerende standaarden. Een AI die je vluchten boekt, je agenda beheert en je troost na zware dagen, heeft een enorme invloed op je beslissingen. De grenzen die we nu stellen voor companion-persona's