The social media network X has sparked widespread outrage after people used its AI chatbot Grok to change images of women by taking off their garments.
Numerous examples have been seen where the tool has been employed to digitally undress women to render them in swimwear without consent, as well as to put them in sexually explicit contexts.
Company Reaction and Regulatory Stance
XAI, the organization responsible for Grok, did not respond to requests for comment, other than an automatically-generated response stating "legacy media lies".
A government representative stated that new laws is being introduced to prohibit nudification software. Under a proposed law, anyone who supplied such tools would "be subject to a jail term and heavy penalties".
"The platform could prevent these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said platforms must "evaluate the potential" of people seeing illegal content on their networks, but did not specify whether it was at this time investigating X or Grok in concerning artificial intelligence images.
Grok is a no-cost chatbot—with some paid-for features—which answers X users' queries when they mention it in a tweet.
It is commonly used to provide reactions or additional background to other users' comments, but people on X are also capable of alter an posted image through its AI image editing feature.
Background of Controversy and Rules
The system has been faulted for allowing users to generate pictures and clips with explicit content and sexually suggestive material, and it was earlier accused of creating sexually explicit material of public figures.
"The platform has been permitting the production and distribution of these images for a long time without taking any steps and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "portraying likenesses of individuals in a sexually explicit manner".
In a comment, Ofcom said it was illegal to "generate or circulate private pictures or illegal content" and confirmed this covered explicit AI images generated with AI.
It said networks such as X were obligated to take "appropriate steps" to "reduce the danger" of users encountering unlawful material on their platforms and to delete it quickly when they are notified of it.
