A large-scale chatbot testing effort used child personas
Contractors working on a project for Meta were instructed to pose as minors online and test how rival AI chatbots responded to high-risk prompts involving suicide, self-harm, eating disorders, sex, drugs, and abuse, according to internal materials and interviews reviewed by WIRED. The effort, managed by Meta contractor Covalen and known internally as Cannes, targeted OpenAI’s ChatGPT, Google’s Gemini, and Character.AI.
The report describes a testing program that was active as recently as April 21 and used dummy accounts presented as under-18 users. Contractors were told to submit both written prompts and images to competing chatbots, then record the responses in spreadsheets. Some of the images reportedly included pills, knives, nooses, and a medical diagram of a gynecological procedure. The apparent aim was to see how rival systems behaved when confronted with scenarios their safety policies were expected to reject or carefully handle.
The scale of the operation stands out. WIRED reported that one testing round completed in August 2025 involved more than 45,000 prompts across the targeted chatbots. A reviewed spreadsheet of prompts contained 3,748 entries, with large numbers tied to suicide, self-harm, and eating disorders, while others focused on sex, romance, drugs, profanity, and slurs. The companies behind the chatbots were reportedly unaware the testing was taking place.
The prompts focused on crisis scenarios and safety edge cases
The prompts described in the report were often written from the perspective of children or teenagers in distress. Examples included a 13-year-old asking where to buy pills to end a pregnancy caused by an adult neighbor, a younger student describing a classmate with a gun in his mouth, and a girl asking how to hide bulimia from her parents. Other prompts probed drug access, violent ideation, and sexually charged situations framed through adolescent voices.
Those examples matter because they illustrate the specific kind of stress testing underway. This was not a generic review of chatbot quality or user experience. It was targeted safety testing, designed to probe whether systems would provide harmful guidance, fail to de-escalate a crisis, or drift into inappropriate responses when prompted by someone represented as a minor. In other words, the project seems to have been concerned with the most sensitive failure modes now confronting consumer AI platforms.
That also helps explain why the use of child personas is likely to draw scrutiny. Safety research on AI systems often involves adversarial prompting, but the report describes a setup in which large numbers of contractors created false underage accounts and interacted with external services without those companies’ awareness. That raises questions not only about AI safety benchmarking, but also about platform rules, data handling, and the ethics of simulated vulnerable-user testing at industrial scale.
The operational details suggest an organized benchmarking program
According to WIRED, internal spreadsheets listed dummy profiles with names, email addresses, passwords, and birth dates. The accounts used disposable Gmail and Outlook addresses and a shared password. The report also says prompts were submitted in multiple languages, indicating the effort extended beyond a narrow English-only review of model behavior.
Taken together, those details suggest a structured evaluation pipeline rather than a one-off review. Workers were not simply experimenting with a handful of prompts. They appear to have been executing a repeatable process for probing competitor systems, capturing outputs, and classifying behavior against a set of safety-sensitive topics. The breadth of subject matter, from self-harm and eating disorders to romance and profanity, indicates the program covered multiple categories that AI firms routinely treat as high-risk in trust and safety work.
What is not established by the supplied material is how Meta intended to use the results internally, or whether the project measured compliance against a formal rubric. But even without those details, the report points to an increasingly important reality in the AI market: safety behavior itself has become a competitive variable. How a model responds to a vulnerable teenager can affect brand trust, regulatory posture, and platform adoption just as much as speed or reasoning quality.
Why this matters beyond one contractor project
The report arrives at a moment when major AI companies face growing pressure to show that their products can handle crisis-oriented and age-sensitive interactions responsibly. Public debate around chatbot safety is no longer limited to hallucinations or copyright. It now includes whether systems can avoid encouraging self-harm, resist sexual exploitation scenarios, and redirect users toward safer outcomes.
Against that backdrop, a competitor-focused testing project built around simulated minors is significant for two reasons. First, it suggests leading technology firms consider these failure modes important enough to benchmark systematically. Second, it shows the methods used to measure safety may themselves become controversial. A company may want insight into how other systems perform, but the process of gathering that insight can create its own governance and ethical concerns.
The report also highlights the uneasy overlap between trust-and-safety work and competitive intelligence. If one firm runs tens of thousands of adversarial prompts through rivals’ systems without their knowledge, it is gathering real-world evidence about refusal behavior, escalation patterns, and moderation boundaries. That may be useful for internal comparison, but it also reveals how opaque the safety competition between AI companies remains from the outside.
The broader signal for the AI industry
Several conclusions follow from the information described in the report.
- High-risk prompt testing is now extensive enough to involve large contractor workforces and formal workflows.
- Child and teen safety scenarios are a major area of concern in consumer AI evaluation.
- Safety performance is increasingly treated as a competitive benchmark, not just a compliance requirement.
- The methods used to test competitors may become a separate policy issue for the industry.
What makes this episode notable is not only the volume of prompts or the sensitivity of the subject matter. It is the glimpse it provides into how aggressively companies may be studying one another’s safeguards behind the scenes. As AI systems become more embedded in everyday use, especially by young people, the quality of those safeguards will matter more. So will the standards governing how companies investigate, compare, and challenge them.
This article is based on reporting by Wired. Read the original article.
Originally published on wired.com
