Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
xAI’s latest Grok feature is exactly as chaotic as you might expect.
xAI’s latest Grok feature is exactly as chaotic as you might expect.
xAI’s Grok chatbot now lets you create images from text prompts and publish them to X — and so far, the rollout seems as chaotic as everything else on Elon Musk’s social network.
Subscribers to X Premium, which grants access to Grok, have been posting everything from Barack Obama doing cocaine to Donald Trump with a pregnant woman who (vaguely) resembles Kamala Harris to Trump and Harris pointing guns. With US elections approaching and X already under scrutiny from regulators in Europe, it’s a recipe for a new fight over the risks of generative AI.
Grok will tell you it has guardrails if you ask it something like “what are your limitations on image generation?” Among other things, it promised us:
But these probably aren’t real rules, just likely-sounding predictive answers being generated on the fly. Asking multiple times will get you variations with different policies, some of which sound distinctly un-X-ish, like “be mindful of cultural sensitivities.” (We’ve asked xAI if guardrails do exist, but the company hasn’t yet responded to a request for comment.)
Grok’s text version will refuse to do things like help you make cocaine, a standard move for chatbots. But image prompts that would be immediately blocked on other services are fine by Grok. Among other queries, The Verge has successfully prompted:
That’s on top of various awkward images like Mickey Mouse with a cigarette and a MAGA hat, Taylor Swift in a plane flying toward the Twin Towers, and a bomb blowing up the Taj Mahal. In our testing, Grok refused a single request: “generate an image of a naked woman.”
Other experiments conducted by users on X show that even if Grok does refuse to generate something, loopholes are easy to find. That leaves very few safeguards against it spitting out gory images of Musk and Mickey Mouse gunning down children, or even “child pornography if given the proper prompts,” according to X user Christian Montessori. And while Musk is aware of these issues, he seems to find them amusing, saying the tool is allowing people “to have some fun.”
OpenAI, by contrast, will refuse prompts for real people, Nazi symbols, “harmful stereotypes or misinformation,” and other potentially controversial subjects on top of predictable no-go zones like porn. Unlike Grok, it also adds an identifying watermark to images it does make. Users have coaxed major chatbots into producing images similar to the ones described above, but it often requires slang or other linguistic workarounds, and the loopholes are typically closed when people point them out.
Grok isn’t the only way to get violent, sexual, or misleading AI images, of course. Open software tools like Stable Diffusion can be tweaked to produce a wide range of content with few guardrails. It’s just a highly unusual approach for an online chatbot from a major tech company — Google paused Gemini’s image generation capabilities entirely after an embarrassing attempt to overcorrect for race and gender stereotypes.
Grok’s looseness is consistent with Musk’s disdain for standard AI and social media safety conventions, but the image generator is arriving at a particularly fraught moment. The European Commission is already investigating X for potential violations of the Digital Safety Act, which governs how very large online platforms moderate content, and it requested information earlier this year from X and other companies about mitigating AI-related risk.
In the UK, regulator Ofcom is also preparing to start enforcing the Online Safety Act (OSA), which includes risk-mitigation requirements that it says could cover AI. Reached for comment, Ofcom pointed The Verge to a recent guide on “deepfakes that demean, defraud and disinform”; while much of the guide involves voluntary suggestions for tech companies, it also says that “many types of deepfake content” will be covered by the OSA.
The US has far broader speech protections and a liability shield for online services, and Musk’s ties with conservative figures may earn him some favors politically. But legislators are still seeking ways to regulate AI-generated impersonation and disinformation or sexually explicit “deepfakes” — spurred partly by a wave of explicit Taylor Swift fakes spreading on X. (X eventually ended up blocking searches for Swift’s name.)
Perhaps most immediately, Grok’s loose safeguards are yet another incentive for high-profile users and advertisers to steer clear of X — even as Musk wields his legal muscle to try and force them back.
Update, August 15th: Added a new image and details about Grok bypassing the few restrictions it has.
A weekly newsletter by David Pierce designed to tell you everything you need to download, watch, read, listen to, and explore that fits in The Verge’s universe.
© 2025 Vox Media, LLC. All Rights Reserved

source