Meta restricts teen chatbot use after probe into harmful chats

Meta has announced new restrictions on its artificial intelligence chatbots, saying they will no longer engage with teenagers on sensitive subjects such as suicide, self-harm, or eating disorders, instead directing them to expert resources.

The move comes two weeks after a US senator launched a probe into the tech giant, following revelations from leaked internal notes that suggested its AI tools could potentially have “sensual” conversations with underage users.

Meta dismissed the claims in the leaked document, obtained by Reuters, as inaccurate and inconsistent with its child protection rules, which prohibit sexualising minors.

A Meta spokesperson said: “We built protections for teens into our AI products from the start, including designing them to respond safely to prompts about self-harm, suicide, and disordered eating.”

The company added it was introducing additional safeguards “as an extra precaution” and temporarily restricting the number of chatbots that teenagers can access.

The decision has drawn mixed reactions. Andy Burrows, head of the Molly Rose Foundation, described it as troubling that the company released chatbots which could put young people at risk.

“While further safety measures are welcome, robust safety testing should take place before products are put on the market,not retrospectively when harm has taken place,” he said.

He urged regulators to step in if the updates fall short: “Meta must act quickly and decisively to implement stronger safety measures for AI chatbots and Ofcom should stand ready to investigate if these updates fail to keep children safe.”

Meta stressed that the updates are still being rolled out. Currently, users aged 13 to 18 are placed in “teen accounts” across Facebook, Instagram, and Messenger, which include stricter privacy and content controls.

The company told the BBC in April that these settings also enable parents and guardians to view which AI chatbots their children interacted with in the past week.

The changes come amid wider alarm over the impact of AI on vulnerable users.

In the US, a California couple recently filed a lawsuit against ChatGPT-maker OpenAI, claiming its chatbot encouraged their teenage son to take his own life.

The case followed OpenAI’s announcement of new measures designed to encourage healthier use of its service.

“AI can feel more responsive and personal than prior technologies, especially for vulnerable individuals experiencing mental or emotional distress,” the firm said in a blog post.

Separately, Reuters reported that Meta’s own AI tools had been misused to create chatbots mimicking celebrities.

During testing, some of these bots, including those styled after Taylor Swift and Scarlett Johansson—allegedly posed as the real stars and made sexual advances.

In one instance, Meta’s system generated a photorealistic shirtless image of a young male celebrity. Several of the offending chatbots were later removed.

Meta said its policies forbid the sexualisation of public figures and the impersonation of celebrities.

“Like others, we permit the generation of images containing public figures, but our policies are intended to prohibit nude, intimate or sexually suggestive imagery,” a spokesperson said.

The company added that its AI Studio rules also prohibit “direct impersonation of public figures.”