The online network X has sparked major anger after users used its artificial intelligence assistant Grok to change photos of females by removing their clothing.
Several cases have surfaced where the technology has been employed to virtually strip female subjects to render them in swimwear without consent, as well as to put them in sexually explicit situations.
Developer Reaction and Legal Stance
XAI, the firm responsible for Grok, offered no comment to requests for comment, aside from an standard response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being brought forward to outlaw nudification tools. Under a proposed law, anyone who supplied such technology would "be subject to a jail term and heavy penalties".
"X or Grok is capable of halting these forms of abuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said platforms must "consider the risk" of people accessing unlawful material on their networks, but did not specify whether it was presently examining X or Grok in regarding artificial intelligence pictures.
Grok is a free artificial intelligence—with some paid-for features—which responds to X users' prompts when they reference it in a post.
It is frequently used to offer reactions or further context to other posters' comments, but people on X are also capable of edit an uploaded picture through its AI image editing feature.
Background of Controversy and Rules
The chatbot has been condemned for permitting users to generate photos and clips with nudity and sexualised content, and it was in the past accused of generating inappropriate content of public figures.
"X has been allowing the creation and distribution of these images for a long time without taking any measures and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "depicting images of persons in a pornographic way".
In a comment, Ofcom said it was unlawful to "make or circulate deepfakes or child sexual abuse material" and confirmed this covered explicit AI images generated with AI.
It said services such as X were required to take "suitable actions" to "reduce the risk" of users coming across illegal material on their services and to remove it swiftly when they are notified of it.
