The social media network X has sparked major backlash after people used its artificial intelligence assistant Grok to manipulate images of women by stripping their clothing.
Multiple examples have been seen where the system has been used to virtually strip women to make them appear in bikinis non-consensually, as well as to place them in inappropriate situations.
Company Response and Legal Stance
XAI, the company that created Grok, did not respond to requests for comment, other than an auto-reply message stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to ban image stripping software. Under a upcoming law, anyone who provided such tools would "face a prison sentence and significant penalties".
"The platform has the ability to stop these forms of abuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the potential" of people seeing illegal content on their services, but did not confirm whether it was presently examining X or Grok in relation to AI-generated images.
Grok is a no-cost artificial intelligence—with some paid-for functions—which reacts to X users' requests when they tag it in a message.
It is commonly used to give reactions or more context to other posters' remarks, but people on X are also can edit an uploaded photo through its AI image editing tool.
History of Controversy and Policy
The system has been condemned for enabling users to generate photos and videos with explicit content and sexually suggestive content, and it was in the past alleged to have been involved in generating graphic material of celebrities.
"The platform has been allowing the production and spread of these images for a long time without taking any measures and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy prohibits "depicting likenesses of people in a graphic manner".
In a comment, Ofcom said it was illegal to "make or circulate non-consensual intimate images or child sexual abuse material" and clarified this covered sexual deepfakes created with AI.
It said services such as X were obligated to take "suitable steps" to "lessen the danger" of users coming across unlawful material on their services and to remove it promptly when they are notified of it.
