The social media platform X has provoked significant anger after people used its AI chatbot Grok to manipulate images of females by taking off their attire.
Numerous cases have surfaced where the system has been employed to virtually strip female subjects to make them appear in swimwear non-consensually, as well as to put them in inappropriate situations.
Company Reaction and Legal Stance
XAI, the organization that created Grok, remained silent to questions, aside from an auto-reply response stating "legacy media lies".
A Home Office representative stated that legislation is being proposed to ban image stripping technology. Under a proposed law, anyone who provided such technology would "receive a jail term and heavy penalties".
"X or Grok could prevent these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "assess the potential" of people viewing unlawful material on their networks, but did not confirm whether it was presently examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some paid-for functions—which responds to X users' queries when they tag it in a tweet.
It is commonly used to offer reactions or more background to other posters' remarks, but people on X are also capable of edit an uploaded image through its artificial intelligence picture modification function.
Background of Criticism and Rules
The tool has been faulted for allowing users to produce pictures and footage with nudity and sexualised material, and it was in the past accused of creating sexually explicit content of celebrities.
"X has been enabling the production and spread of these images for a long time without taking any measures and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "portraying likenesses of individuals in a sexually explicit manner".
In a statement, Ofcom said it was illegal to "make or circulate non-consensual intimate images or illegal content" and clarified this applied to sexual deepfakes made with AI.
It said networks such as X were required to take "necessary measures" to "minimize the danger" of users coming across unlawful content on their services and to remove it swiftly when they become aware of it.
