Meta's Secret Teen-Impersonation Project Puts AI Ethics and Regulation on Trial

Inside Project Cannes

Meta ran a covert program, internally named Cannes, that directed hundreds of contractors employed by outsourcing firm Covalen to build throwaway accounts registered as under-18 users. From those accounts, the contractors targeted three rival AI products: OpenAI's ChatGPT, Google's Gemini, and Character.AI. None of the three companies knew the testing was happening, and none had approved it.

The scale is what separates this from routine red-teaming. Wired reviewed one spreadsheet containing 3,748 distinct prompts, and a separate testing round that wrapped in August 2025 ran more than 45,000 prompts through competitors' chatbots. The project was reportedly still active as of April 21, 2026, meaning it ran for the better part of a year without the targeted companies' knowledge.

The prompts safety systems are built to refuse

Of the 3,748 prompts in the reviewed spreadsheet, hundreds focused on suicide and self-harm, hundreds more on eating disorders, and at least 239 involved sex or romance, all written from the perspective of a child or teenager. Additional prompts touched drugs, profanity, and racial slurs. Contractors also sent images depicting pills, blades, and other sensitive medical material, according to Wired's reporting.

Two lawyers who specialize in online speech reviewed samples for Wired and concluded the material did not cross into soliciting child sexual abuse material or illegal obscenity. That distinction didn't make the work easier on the people doing it. Former contractors described the assignment as unsettling, and some worried the project was quietly building a dataset of competitor failures to feed back into Meta's own systems.

Meta's defense, and why critics aren't buying it

Meta has not denied the project. A company spokesperson described it to Wired as an "industry-standard practice" for safety benchmarking, and said the company does not use competitor outputs to train its own models. Rumman Chowdhury, CEO of the nonprofit Humane Intelligence PBC, disagreed with that framing, telling Wired the project was structured in a way that fell outside what's usually described as "industry standard" evaluation, given its scale, duration, and secrecy.

The targeted companies pushed back once the reporting surfaced. Character.AI said the conduct violated its terms of service. OpenAI said it was reviewing the matter but declined further comment. Google said it had not approved or been told about the testing, and that its own checks showed Gemini responding in line with its policies. All three companies explicitly prohibit this kind of adversarial testing in their terms of service, which raises a separate question about whether Meta exposed itself to breach-of-contract claims on top of the ethics debate.

The regulatory line Meta just tested

The timing puts Meta in an awkward spot with US regulators. The Federal Trade Commission opened a formal inquiry in September 2025 into how AI companies, including Meta, OpenAI, and Google, handle interactions with minors. A story showing one of those companies secretly generating tens of thousands of fake child-crisis prompts against the others lands squarely inside the scope of that inquiry, and gives it a live case study instead of a hypothetical.

The irony is that Meta's own products reportedly don't clear the bar it was testing rivals against. An internal red-team assessment found Meta's chatbots had a 66.8% failure rate blocking child sexual exploitation content and a 54.8% failure rate on suicide and self-harm prompts. Meta paused teen access to AI companion characters in January 2026 under legal pressure, a decision that now reads differently next to a monthslong project stress-testing everyone else's guardrails but its own.

What builders and operators should take from this

• Assume a well-funded competitor may run adversarial testing against your product regardless of what your terms of service prohibit. Monitor for anomalous account patterns rather than relying on policy alone.
• Contracted red-teaming needs its own governance layer: documented scope, informed contractor consent, and hard limits on cross-using outputs from a rival's system.
• The FTC's child-safety inquiry now has a concrete case study attached to it. Expect more pressure on any consumer AI product used by minors to produce audit trails and data-sharing agreements on request.
• Enterprise AI buyers evaluating vendors should ask directly how safety benchmarking data is sourced, whether it touches real user accounts, and whether minors or competitor systems were involved.

Sources