India, France, Germany: List of nations cracking down on xAI’s Grok over deepfake abuse

Grok, the AI chatbot developed by Elon Musk-owned xAI, is yet again drawing the ire of regulators around the world amid growing backlash over the AI chatbot generating sexualised, non-consensual images of women and underage girls.

After weeks of intense criticism and scrutiny, xAI announced on Tuesday, January 6, that it has raised $20 billion in its latest funding round featuring prominent investors such as Nvidia, Fidelity Management and Resource Company, Qatar’s sovereign wealth fund, and Valor Equity Partners.

The Series E funding round reportedly exceeded the AI startup’s initial $15 billion target, according to a press release by xAI in which it also touted Grok’s image-generation feature known as Grok Imagine as “lightning-fast” with “state-of-the-art multimodal understanding”

Towards the end of December 2025, several users on X began commenting on images of people – usually women – by tagging Grok and asking the chatbot to “put her in a bikini” or “take her dress off”. The AI chatbot, which auto-replies when tagged, complied with these user requests and generated non-consensual sexual images of both celebrities and non-celebrities, including some who appeared to be young children.

In response, xAI has sought to shift the liability squarely on users, warning that those who submit prompts asking Grok to generate illegal content will face the same consequences as those uploading illegal content on X.

“We take action against illegal content on X, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary,” the platform’s safety team further said.

However, the list of countries contemplating regulatory action against X has continued to grow. Here’s a look at how regulators across different regions have responded to the latest Grok incident. Notably, lawmakers and regulatory bodies in the United States, where xAI is headquartered, are yet to comment on the issue.

Malaysia’s digital regulator said that it is “presently investigating the online harms in X.”

In a statement, The Malaysian Communications and Multimedia Commission said that it has “taken note with serious concern of public complaints about the misuse of artificial intelligence (AI) tools on the X platform, specifically the digital manipulation of images of women and minors to produce indecent, grossly offensive, and otherwise harmful content.”

Three French government ministers have reportedly flagged Grok’s outputs to the Paris prosecutor’s office, which has said that it will investigate the proliferation of sexually explicit deepfakes on X. The country’s digital affairs office said that the ministers also reported the “manifestly illegal content” to a government online surveillance platform in order “to obtain its immediate removal”, according to a report by Politico.

Regulators have also been asked to decide whether the images violate the European Union’s Digital Services Act (DSA), the legislative framework for governing the handling of content by online intermediaries such as X.

Taking note of Grok being abused to generate objectionable pictures and videos of women, without their consent, the IT Ministry said it has sent a directive to X which accused the social media platform of not adhering to the country’s laws and raised red flags over X’s “serious failure” to enforce safeguards.

“Grok AI developed by you and integrated and made available on the X platform, is being misused by users to create fake accounts to host, generate, publish or share obscene images or videos of women in a derogatory or vulgar manner in order to indecently denigrate them,” the IT Ministry said in the notice sent to the social media company’s chief compliance officer on Friday, January 2.

The IT Ministry directed X to submit a detailed action taken report by Wednesday, January 7, covering specific technical and organisational measures adopted by X in relation to Grok AI, the role and oversight exercised by its India chief compliance officer, and action taken by the company against offending content, users and accounts.

It also directed the company to undertake a comprehensive technical, procedural and governance-level review of Grok AI, including its prompt-processing, output generation, and image handling and safety guardrails, “to ensure that the application does not generate, promote or facilitate content which contains nudity, sexualisation, sexually explicit or otherwise unlawful content”.

The European Commission on Monday, January 5, said that the images of undressed women and children being shared across e were unlawful and appalling. Stating that the Commission was aware of the fact that X was offering a ‘spicy mode’ to users, European Commission spokesperson Thomas Regnier said “This is not spicy. This is illegal. This is appalling. This is disgusting. This is how we see it, and this has no place in Europe.”

British regulator Ofcom on Monday said it has asked X to explain how Grok was able to produce undressed images of people and sexualised images of children, and whether it was failing in its legal duty to protect users.

“We have made urgent contact with X and xAI to understand what steps they have taken to comply with their legal duties to protect users in the UK,” a spokesperson said. Liz Kendall, the UK’s technology secretary, also condemned Grok’s deepfakes on Tuesday, January 6.

Creating or sharing non-consensual intimate images or child sexual abuse material, including AI-generated hyper-realistic sexual imagery, is illegal in Britain, according to a report by Reuters. Additionally, social media platforms are legally obligated to take steps to stop users from encountering illegal content and take it down when they become aware of it.

Germany is the latest country to call on the EU to take legal action against Musk’s X platform over Grok-generated, sexually explicit deepfakes. German media minister Wolfram Weimer urged the European Commission on Tuesday, January 6, to take legal action to stop what he called the “industrialisation of sexual harassment” taking place on X.

“What we are currently observing on X looks like the industrialisation of sexual harassment. It is now crucial that the EU Commission continues to enforce this (legal framework) as rigorously as it has already begun,” Weimer was quoted as saying by Reuters.

Germany’s digital ministry, which is responsible for DSA enforcement through the federal network regulator, said it was committed to ensuring compliance with the law and urged everyone to use its reporting rights.

“The challenge at present is primarily to enforce the various – in some cases new – rights more consistently and to actually make use of them. Anyone who creates or distributes such images without consent is committing a serious violation of personal rights and may be liable to prosecution in many cases,” a ministry spokesperson was quoted as saying by Reuters.