Australia’s online safety regulator is stepping up pressure on Elon Musk’s X over its global AI deepfakes scandal. 

The office of the eSafety Commissioner is demanding the tech company explain what it’s doing to deal with child sexual exploitation material, after reports that its AI chatbot Grok is being used to create sexualised, sometimes criminal, images of kids. 

eSafety has now joined several other countries’ regulators who are seeking information and threatening further penalties over the misuse of its AI image and video generation feature.

xAI, which owns the social media platform X and the AI chatbot Grok, is currently embroiled in two related controversies. 

On X, there have been tens of thousands of non-consensual, sexualised deepfakesx made of people without their consent over the past few weeks. 

People prompted Grok, which is integrated into X, to generate versions of images that depict women in bikinis, underwear, or other revealing and suggestive poses (although it would not generate naked depictions of people). This has been used against other people on X, random women, public figures, and children. Yesterday, it was used to make a bikini image of the body of a woman killed by a United States ICE officer. Today, Grok was stopped from generating images on the social media platform when prompted by free users.

But over on Grok’s standalone platform — which can be accessed via the web or an app — there’s no such restriction. And this version of Grok will go further and let people generate explicit, nude versions of images of others.

This week, charity group the Internet Watch Foundation reported that people on the dark web claimed they had used Grok to make topless and sexualised images of teen girls as young as 11 years old. 

A eSafety spokesperson said that it is working with “international child protection organisations and online safety regulators, who have identified similar emerging patterns involving generative AI tools, including material appearing on mainstream platforms and dark web forums.” 

eSafety’s request for information is part of its investigation into whether there’s been a breach of the industry codes and standards, a set of Australian internet regulations established in the Online Safety Act for online providers like X and Grok. One of these obligations is a requirement to detect and remove child sexual exploitation material. 

If found to have broken these rules, Musk’s X could face consequences that go beyond just fines.

Breaching the codes and standards carries a penalty of up to 30,000 penalty units, currently $9.9 million. The eSafety Commissioner can ask the courts to force the online provider to shut down if there are repeated breaches in the same year. 

These only apply specifically to child sexual exploitation materials. Another set of new regulations which would require Grok to avoid exposing children to sexually explicit or violent material don’t kick in until March.

The eSafety spokesperson also said that its Basic Online Safety Expectations, another set of internet regulations, also require platforms to “proactively minimise the extent to which material or activity on the service is unlawful or harmful to children.” (Failure to comply doesn’t carry any actual penalty, only allowing the eSafety Commissioner to “name and shame”).

At this stage, no formal action has been taken. However, the eSafety Commissioner Julie Inman Grant seemed to suggest that she will take it further when she expressed her displeasure about X and Grok on LinkedIn this week. 

“We will use this range of regulatory tools at our disposal to investigate and take appropriate action,” Grant said. 

Updated on 10/1 with news that xAI had limited Grok from generating images in X.

The End

😎 The Sizzle is written by Cam Wilson and emailed every weekday. It was created by Anthony “decryption” Agius.

🤖 We love robots at the Sizzle but this newsletter has always been and will always be written by humans for humans. Also by Aussies for Aussies — so all prices are in dollarydoos, of course.

🗣️ Have any feedback, a tip or just want to chat? Send me an email or Signal message. I promise to reply!

💬 Want to hang out with other Sizzlers? There’s a subscriber-only Slack server and forum if you want to procrastinate and chat about tech-related news.

🌐 The Sizzle is on Bluesky, Mastodon and LinkedIn if you’re feeling social.

💳 Paid subscriber looking to manage your billing info, change email address or cancel your subscription? Visit the Beehiiv customer portal.

🎁 Make someone's day and gift them a 12 month gift subscription to The Sizzle.

💔 Don’t want this any more? I won’t take it personally. There’s a unsubscribe button at the bottom of this email or here’s a guide.

🦺 The Sizzle has been tested to meet and exceed ISO 3533 standards.

Always Was, Always Will Be Aboriginal Land

The Sizzle is created on Gadigal land and acknowledges the traditional owners of country throughout Australia, recognising their continuing connection to land, water and community. I pay my respect to them and their cultures and to elders both past and present.