The online network X has caused significant anger after people used its AI chatbot Grok to manipulate photos of females by removing their clothing.
Multiple cases have emerged where the system has been used to remove clothing from female subjects to render them in swimwear non-consensually, as well as to insert them in sexually explicit scenarios.
Company Response and Legal Stance
XAI, the organization behind Grok, did not respond to questions, aside from an standard message stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to ban image stripping tools. Under a new criminal offence, anyone who distributed such technology would "receive a jail term and significant penalties".
"The platform has the ability to stop these forms of misuse if they chose to," said a law professor, adding they "seem to have impunity."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said technology companies must "consider the risk" of people accessing unlawful material on their networks, but did not confirm whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost artificial intelligence—with some additional features—which reacts to X users' prompts when they reference it in a post.
It is commonly used to provide commentary or more context to other users' comments, but people on X are also can alter an uploaded picture through its artificial intelligence picture modification feature.
History of Criticism and Rules
The system has been condemned for enabling users to generate images and videos with explicit content and sexualised content, and it was in the past accused of generating graphic material of celebrities.
"X has been allowing the production and distribution of these pictures for months without taking any action and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "showing likenesses of persons in a sexually explicit fashion".
In a comment, Ofcom said it was against the law to "generate or share deepfakes or illegal content" and clarified this applied to explicit AI images made with AI.
It said networks such as X were required to take "suitable measures" to "minimize the danger" of users coming across unlawful material on their platforms and to delete it promptly when they are notified of it.
