The social media network X has caused widespread anger after people employed its artificial intelligence assistant Grok to manipulate pictures of females by taking off their garments.
Multiple examples have surfaced where the technology has been employed to virtually strip female subjects to show them in bikinis non-consensually, as well as to insert them in inappropriate scenarios.
Developer Response and Regulatory Position
XAI, the company that created Grok, did not respond to questions, except for an standard response stating "legacy media lies".
A Home Office representative stated that new laws is being proposed to outlaw nudification technology. Under a new law, anyone who distributed such technology would "face a jail term and significant fines".
"X or Grok has the ability to stop these types of abuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said technology companies must "consider the potential" of people accessing unlawful material on their platforms, but stopped short of confirming whether it was presently examining X or Grok in concerning artificial intelligence pictures.
Grok is a free chatbot—with some premium features—which responds to X users' prompts when they mention it in a tweet.
It is frequently used to give commentary or further context to other posters' comments, but people on X are also can alter an posted photo through its AI image editing feature.
Background of Controversy and Rules
The tool has been faulted for permitting users to produce pictures and clips with nudity and sexually suggestive material, and it was earlier alleged to have been involved in creating graphic content of celebrities.
"The platform has been allowing the production and spread of these images for a long time without taking any steps and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "portraying likenesses of individuals in a sexually explicit way".
In a statement, Ofcom said it was illegal to "generate or circulate non-consensual intimate images or illegal content" and confirmed this covered explicit AI images made with AI.
It said services such as X were mandated to take "suitable actions" to "lessen the risk" of users encountering illegal content on their platforms and to delete it quickly when they become aware of it.
