Elon Musk's social media platform X has provoked widespread outrage after users employed its AI chatbot Grok to manipulate images of females by stripping their garments.
Numerous instances have been seen where the system has been employed to digitally undress female subjects to make them appear in swimwear non-consensually, as well as to put them in sexualized scenarios.
Developer Response and Legal Position
XAI, the organization responsible for Grok, offered no comment to questions, other than an automatically-generated response stating "traditional news outlets lies".
A Home Office official stated that new laws is being proposed to ban nudification technology. Under a proposed law, anyone who provided such technology would "receive a prison sentence and heavy penalties".
"The platform has the ability to stop these types of misuse if they chose to," said a law professor, adding they "seem to have impunity."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "evaluate the potential" of people accessing illegal content on their platforms, but did not specify whether it was presently investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost AI assistant—with some paid-for features—which answers X users' prompts when they mention it in a tweet.
It is frequently used to provide reactions or more background to other users' remarks, but people on X are also capable of edit an posted image through its artificial intelligence picture modification tool.
History of Criticism and Policy
The system has been condemned for permitting users to create photos and clips with explicit content and sexually suggestive material, and it was previously alleged to have been involved in creating sexually explicit material of public figures.
"X has been permitting the production and distribution of these images for a long time without taking any action and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was against the law to "make or distribute deepfakes or illegal content" and confirmed this covered sexual deepfakes generated with AI.
It said services such as X were obligated to take "necessary actions" to "minimize the danger" of users viewing unlawful content on their platforms and to remove it swiftly when they become aware of it.
