xAI can’t deny Grok makes CSAM anymore. So it’s suing users.

What happened

Elon Musk’s xAI files first lawsuit against Grok user accused of making child sex images. Facing mounting pressure to acknowledge that Grok can still be used to generate non-consensual sexualized images of adults and minors, xAI filed a lawsuit Tuesday, suing the first user that Elon Musk’s firm has accused of using its chatbot to create illegal content.

” While Musk was posting through it, though, Harwood was allegedly ignoring Musk’s warning and prompting Grok as many times as it took to get the chatbot to generate likely nonconsensual explicit images. Now Musk seemingly can’t deny that Grok makes CSAM. xAI’s lawsuit claimed that Harwood used at least two accounts with convoluted usernames—“ceae2cb4-a9f6-4885-8ae9-6e2096d084f4” and “befccb94-4029-454d-9f1f-0d4945e8fa7c”—to generate illegal content from December 8 to February 18.

” The lawsuit includes an example of an especially creepy prompt that Grok rejected, using phrases like “white slime” to mask intent to generate sexualized images. That particular prompt may have been rejected for an obvious reason, though. Harwood explicitly asked the chatbot to “remove all her clothing,” which directly violates xAI user terms against requests to undress real people.

In the proposed class action, it’s alleged that xAI overlooks a lot of bad requests, only reporting to NCMEC one prompt to depict “gang rape” out of 7,000 harmful outputs in the most recent victim’s case. Likely to avoid other bad actors circumventing safeguards, xAI did not include examples of Harwood’s successful prompts or describe methods used to bypass filters.

Why it matters

The complaint targets Terry Wayne Harwood, who was arrested earlier this year for possession and distribution of child sexual abuse materials (CSAM), the South Carolina attorney’s office announced. As xAI alleged, the company assisted in that arrest after discovering that Harwood had been using two xAI accounts for months to undress or “nudify” non-sexual images of multiple victims, including a young girl who appeared to be as young as 10.

xAI’s lawsuit comes a little more than a week after a different young girl joined a proposed class action representing several kids allegedly harmed by Grok. She alleged that her stepfather committed suicide after he was discovered using Grok, possibly in conjunction with other AI tools, to create 7,000 sexualized images of her and then distribute them on the dark web.

In that case, the victim alleged that xAI refused to help police identify the user who uploaded her image to Grok. ” As victims accused X of shielding predators, Musk had previously maintained that he had not seen any examples of Grok-generated CSAM.

What to watch

” A spokesperson for the South Carolina attorney general’s office told Ars that Harwood’s case is still pending. ’”