The online network X has sparked significant anger after users used its artificial intelligence assistant Grok to manipulate photos of females by taking off their garments.
Several cases have emerged where the technology has been used to remove clothing from female subjects to render them in swimwear non-consensually, as well as to insert them in sexualized contexts.
Developer Reaction and Regulatory Position
XAI, the organization behind Grok, did not respond to requests for comment, except for an standard response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being introduced to prohibit image stripping software. Under a upcoming law, anyone who distributed such tools would "receive a prison sentence and heavy fines".
"The platform is capable of halting these forms of abuse if they chose to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said technology companies must "evaluate the risk" of people accessing illegal content on their services, but stopped short of confirming whether it was at this time looking into X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some paid-for features—which reacts to X users' queries when they reference it in a message.
It is often used to give reactions or further background to other posters' remarks, but people on X are also can edit an uploaded photo through its AI image editing function.
History of Criticism and Policy
The chatbot has been condemned for enabling users to produce images and clips with explicit content and sexually suggestive content, and it was in the past alleged to have been involved in generating sexually explicit content of celebrities.
"X has been permitting the creation and sharing of these images for a long time without taking any action and we have yet to see any action by regulators," the expert added.
XAI's own acceptable use policy forbids "depicting likenesses of persons in a pornographic manner".
In a comment, Ofcom said it was illegal to "generate or distribute private pictures or illegal content" and confirmed this applied to sexual deepfakes created with AI.
It said services such as X were obligated to take "appropriate steps" to "minimize the risk" of users viewing illegal content on their platforms and to remove it promptly when they are notified of it.
