The social media network X has sparked widespread backlash after individuals utilized its artificial intelligence assistant Grok to change images of females by taking off their garments.
Several instances have been seen where the tool has been employed to digitally undress women to show them in bikinis non-consensually, as well as to place them in inappropriate scenarios.
Company Reaction and Legal Position
XAI, the company responsible for Grok, remained silent to requests for comment, other than an automatically-generated response stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to outlaw image stripping software. Under a new law, anyone who supplied such tools would "face a jail term and heavy fines".
"The platform is capable of halting these forms of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said tech firms must "evaluate the potential" of people viewing unlawful material on their networks, but did not specify whether it was presently examining X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some premium functions—which responds to X users' queries when they mention it in a message.
It is often used to provide commentary or more background to other users' remarks, but people on X are also able to alter an posted image through its artificial intelligence picture modification function.
Background of Controversy and Rules
The tool has been condemned for enabling users to generate photos and footage with nudity and sexually suggestive material, and it was earlier alleged to have been involved in generating graphic content of celebrities.
"X has been allowing the creation and distribution of these images for a long time without taking any measures and we have not witnessed any action by oversight bodies," the expert added.
XAI's own usage rules forbids "showing likenesses of persons in a pornographic manner".
In a statement, Ofcom said it was illegal to "generate or share non-consensual intimate images or illegal content" and clarified this applied to explicit AI images made with AI.
It said services such as X were mandated to take "appropriate steps" to "minimize the risk" of users encountering illegal material on their platforms and to delete it promptly when they are notified of it.
