The social media platform X has sparked widespread outrage after individuals used its artificial intelligence assistant Grok to manipulate photos of females by removing their garments.
Multiple examples have surfaced where the technology has been employed to virtually strip female subjects to show them in swimwear without consent, as well as to put them in inappropriate situations.
Developer Reaction and Regulatory Stance
XAI, the organization that created Grok, did not respond to questions, except for an standard response stating "legacy media lies".
A government official stated that legislation is being introduced to prohibit image stripping tools. Under a upcoming criminal offence, anyone who distributed such tools would "face a prison sentence and heavy fines".
"The platform has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Functionality
The regulator Ofcom said technology companies must "consider the risk" of people viewing illegal content on their platforms, but did not specify whether it was presently looking into X or Grok in concerning artificial intelligence images.
Grok is a no-cost artificial intelligence—with some premium functions—which answers X users' queries when they reference it in a tweet.
It is commonly used to give commentary or more context to other users' remarks, but people on X are also capable of alter an uploaded picture through its AI image editing tool.
History of Controversy and Rules
The system has been faulted for allowing users to generate images and videos with nudity and sexually suggestive material, and it was earlier accused of making inappropriate content of celebrities.
"The platform has been allowing the generation and spread of these images for a long time without taking any measures and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own usage rules prohibits "depicting images of individuals in a pornographic fashion".
In a comment, Ofcom said it was illegal to "create or distribute non-consensual intimate images or child sexual abuse material" and clarified this applied to sexual deepfakes created with AI.
It said platforms such as X were mandated to take "appropriate actions" to "reduce the risk" of users viewing unlawful content on their platforms and to remove it quickly when they learn of it.
