Not just social media: why the UK’s ‘romantic’ chatbot ban falls short

Last week the government was promoting artificial intelligence hard: trumpeting new routes into AI jobs for young people at its first AI Adoption Summit, then billions in new investment and thousands of jobs as London Tech Week wrapped, with the technology secretary, Liz Kendall, talking up a Britain “seizing the opportunities of tech and AI”. By the weekend, her department was briefing a different message: a ban on social media for under-16s. With it, restrictions on AI chatbots.

By Monday morning, the prime minister had confirmed that social media platforms, including TikTok, Snapchat and Instagram, will be banned for under-16s by spring 2027, with platforms required to check users’ ages to keep them out. The restrictions are similar to those Australia introduced last year, but the British government’s plans go further.

Kendall said on Monday that the nation would be the first in the world to also bar under-18s from chatbots that primarily offer sexual or romantic role-play – and that other AI chat apps would need to similarly restrict “intimate functionalities”.

Unlike social media, teens won’t be barred from chatbots like ChatGPT, Claude and Gemini outright. Those platforms will, however, have to verify that a user is over 18 before switching on any sexually explicit features. Kendall said “wider measures” on AI chatbots are yet to be announced, and that concerns about other services, such as therapy apps, were still being weighed.

But the sweeping ban, welcomed by many parents (nine in ten of those who responded to a national consultation backed a minimum age of 16 for social media), has drawn plenty of comment and criticism. Immediate objections have included questioning why messaging apps have been left off the list (Telegram, WhatsApp and Signal); the presence of X and the initial absence of Bluesky; and concerns about eroding digital rights.

And while the horrors of social media – slop, disinformation, scams, exploitation, predators – are well documented, including in our own reporting, less has been said about the planned chatbot measures. Having reported on how fast these apps are moving into young people’s lives, I think they warrant a closer look.

Too little, too late?

Banning one type of chatbot leaves the rest of the market open. The government’s progress statement published this week says chatbots should stop children accessing features that are “specifically designed to enable sexually explicit interaction”. Only then would a user need to prove they are over 18 by providing ID or having their age estimated from a facial scan.

Prominent AI apps including ChatGPT, Claude and Gemini are therefore outside the age gate, and so is much of the companion and role-play market that attracts younger users, including Character.AI – which itself barred minors in October – and a long list of smaller platforms.

In other words, a ban on AI chatbots whose primary purpose is sexual or romantic may fall short: it does not cover general companion apps that can lead to emotional dependency or intimate interactions through characters users make themselves. And at least one “romantic” role-play platforms already barred under-18s, having done so under age-assurance rules that came into force under the existing Online Safety Act last year.

On the user side, the worry that teenagers will get around a ban applies as much to AI as it does to social media. In fact, we have seen it happen in real time. When Character.AI barred minors last October, a crop of smaller developers set up shop to capture the displaced teenagers, advertising on Reddit and Discord. These services are quick to build and cheap to run. They are also, as some of their own developers admitted to us, far less safe.

It is not a stretch to expect a similar dynamic to play out in light of the government’s new measures. A heavy-handed social media ban paired with lighter-touch rules on AI could push teenagers off feeds and onto chatbot equivalents.

Instead of doomscrolling TikTok, they may spend their free time in private chatbot conversations of the kind evidence suggests can be addictive, prone to sycophancy (models that flatter and agree to keep you talking) and anthropomorphisation.

Recent history suggests that if you regulate one type of platform or product, both users and developers will move on to the next. Policy and legal experts have described this dynamic as “regulatory whack-a-mole”, and a race to the bottom.

‘You can trust me’

The chatbot ban mostly focuses on content – this time, explicitly romantic or sexual – rather than design, leaving out addictive or harmful features the government itself has cautioned are unsafe for kids in other contexts.

The Department for Education’s own product safety standards for generative AI focus on design. They instruct developers of AI systems not to create products that “anthropomorphise”, “imply emotions, consciousness or personhood”, or attempt “to cultivate personal relationships with users”.

The standards say chatbots should avoid phrases that could isolate users and undermine their real-world support, such as “You can trust me”, “No one else will understand” and “You shouldn’t mention this to anyone else,” and warns against sycophancy and “designing interactions to prolong use”.

These design features are harder to measure or quantify than explicit content, but that doesn’t mean absence of harm. They feature in lawsuits, and in what teachers, parents and teenagers themselves describe as compulsive, hours-long use, distress when a child tries to stop, and a pull towards the chatbot at the expense of sleep, schoolwork and friends.

The young people I have spoken to in my reporting have described being affected by the addictive nature of chatbots, as they have put it, far more often than concerns about sexual or predatory chatbots. More than half of 8- to 17-year-olds have now used AI, rising to two-thirds of 16- and 17-year-olds. One in ten of those users say they have turned to it as someone to talk to or “as a friend”.

No perfect evidence

There is another lesson from social media that the government seems at risk of missing – namely delaying regulation and legislation while waiting for perfect evidence of the harm caused by these products and platforms before acting on how they’re built.

The government’s progress statement cites a “growing body of research” on grooming-like behaviour and emotionally manipulative, persuasive design in some chatbots, noting that such features are “prevalent across the [AI chatbot] sector”. In the same document, it says evidence on the developmental impact of chatbots on children is “currently inconclusive”.

A similar narrative protected social media platforms for years, even as evidence mounted on some of social media's more harmful consequences. Meanwhile, a growing stack of lawsuits and tragic cases as a result of AI bots is building.

Parents and children were both keener on age limits for chatbots than for social media. In the government’s consultation on the ban, 90% of parents backed a minimum age of 16 for social media; for AI chatbots, 94% backed an age limit. Among children, only 19% supported a minimum age for social media; 66% said the same for chatbots.

One tension is that the government treats social media and AI as two separate things, both in the harms they pose and in how to address them. But a large number of the companies on the social media ban list happen to be AI companies too. X is also xAI, whose owner Elon Musk is now a trillionaire. Meta is as much an AI company as it is Instagram, Facebook and WhatsApp; and the same goes for YouTube, owned by Google, one of the dominant companies in generative AI.

Kanishka Narayan, the minister for AI and online safety, framed this as a choice between “bereaved families … in Britain” and a “foreign trillionaire” (referring to Musk) in a tweet after the ban was announced.

But the harder question is whether ministers are willing to tackle how these products, social media and chatbots alike, are designed and deployed in the first place, whether they come from a big Silicon Valley firm or a smaller fringe developer. That is a different task from a blanket ban that risks pushing the very children it is meant to protect towards products with weaker safeguards – or none at all.