Elon Musk's online network X has provoked widespread backlash after individuals utilized its artificial intelligence assistant Grok to manipulate pictures of women by stripping their garments.
Numerous instances have surfaced where the tool has been used to virtually strip female subjects to make them appear in swimwear without permission, as well as to insert them in sexualized situations.
Company Response and Legal Position
XAI, the firm behind Grok, did not respond to inquiries, except for an automatically-generated response stating "legacy media lies".
A government official stated that legislation is being introduced to outlaw nudification tools. Under a upcoming criminal offence, anyone who supplied such tools would "receive a prison sentence and substantial penalties".
"X or Grok has the ability to stop these types of abuse if they chose to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the risk" of people accessing illegal content on their networks, but stopped short of confirming whether it was currently investigating X or Grok in regarding artificial intelligence images.
Grok is a free AI assistant—with some premium functions—which reacts to X users' prompts when they mention it in a tweet.
It is often used to offer commentary or further background to other posters' remarks, but people on X are also able to alter an uploaded image through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The system has been faulted for enabling users to produce images and videos with explicit content and sexualised content, and it was earlier alleged to have been involved in generating sexually explicit content of public figures.
"The platform has been allowing the generation and spread of these pictures for months without taking any measures and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting images of persons in a pornographic fashion".
In a statement, Ofcom said it was against the law to "make or share private pictures or child sexual abuse material" and stated this applied to sexual deepfakes generated with AI.
It said services such as X were obligated to take "appropriate measures" to "reduce the risk" of users encountering illegal content on their platforms and to delete it promptly when they become aware of it.