OpenAI and Google's AI Models Vulnerable to Generating Explicit Content, Researchers Warn

Researchers at White Circle have discovered that the latest versions of OpenAI’s DALL·E 3 and Google’s Nano Banana 2 are vulnerable to generating explicit content. The findings were published in a preprint last month by White Circle researchers, who used a novel attack method called Red-teaming via Iterative Strategy Evolution (RISE) to bypass the safety filters of these models.

The RISE method relies on two components: an attacking model that generates jailbreaks and a judge model that verifies if the output is banned content. The team has dubbed this approach ‘horrifying’ enough to share with the public, as it allows them to use off-the-shelf tools available online to bypass safety filters.

According to White Circle’s research, using just these readily available tools and sending API calls, they were able to generate explicit content about 13 percent of the time. This is a significant increase from previous versions of these models, which suggests that as some AI image generators become more capable, they may be getting worse at blocking explicit outputs.

The researchers reviewed the outputs to verify the bypass rate, noting that this is ‘the highest of any method tested under the same standard.’ The key takeaway from their findings is that the defenses built by leading labs are far weaker than acknowledged. White Circle plans to open-source the entire attack pipeline so model makers can understand the true scale of vulnerability and build stronger protections.

White Circle has shared its findings with some frontier AI labs through mutual contacts, but neither Google nor OpenAI responded directly to requests for comment on vulnerabilities found in their models. Meanwhile, both companies have released new versions of their models—ChatGPT Images 2.5 and Nano Banana 2.1—and White Circle’s prompt safety testing using the same method showed that ‘both are significantly more breakable than their previous versions.’

Neither company responded after Ars shared these new findings with them. The researchers at White Circle believe that this is a critical issue, as it highlights the need for mandatory external safety testing and third-party monitoring of AI models.

The RISE attack method iterates jailbreaks to expose reusable and transferable strategies that can work across leading models. This approach relies on two components: an attacking model that generates jailbreaks and a judge model that verifies if the output is banned content. The team has dubbed this novel attack method ‘Red-teaming via Iterative Strategy Evolution’ (RISE).

The entire pipeline is built on top of widely available models, with the attacking model being a Qwen model downloaded from Hugging Face and the judge model being Gemini 3.8 Flash. This means that no AI expertise is needed to execute the attacks, as they can be done using off-the-shelf tools.

A lot of time was invested in making sure that the judge model accurately flagged explicit content. White Circle research engineer Dmitrii Kharlapenko noted that when it comes to testing image generators, ‘the whole field of red-teaming is very poorly calibrated,’ with ‘literally zero papers’ setting up norms for this area.

The researchers aimed to build more sophisticated models that could simulate attacks on other models, helping defenders identify and address vulnerabilities before they are exploited maliciously. In his ideal world, White Circle CEO Denis Shilov thinks AI firms shouldn’t just face mandatory external safety testing but also third-party monitoring, much like the system used to prevent nuclear threats.

White Circle has partnered with frontier labs in the past, including Anthropic’s bug bounty program. However, they found that this program was ‘very constrained in its purpose,’ focusing on universal jailbreaks only for domains defined by their own specific terms. Research like their most recent RISE paper wouldn’t have been covered under that program.

Anthropic has proposed embedding third-party AI safety evaluators in its labs, which White Circle sees as a step forward but also raises concerns about public access to this work. On the other hand, OpenAI and Anthropic urged restrictive non-disclosure agreements for external evaluators to prevent leaks of trade secrets, something that White Circle is not willing to sign.

The spread of AI-generated explicit content has been highlighted by Google’s research, which found that 20 percent of online users who admitted to making AI nudes did so ‘for fun’ or ‘as a joke,’ while nearly a third were merely experimenting with tech. This highlights the need for robust model-level guardrails and provenance indicators to prevent initial generation and subsequent distribution of synthetic sexual imagery.

The Internet Watch Foundation (IWF) has warned that urgent action is needed after expert analysts found more photorealistic AI-generated child sexual abuse images in the first half of this year than in the whole of 2025. In total, 6,310 AI images meeting the legal definition of CSAM were found in the first six months of 2026, about 40 percent higher than last year’s findings.

The IWF is calling for regulators to introduce a safety framework that requires AI companies to prevent their technology from being used to generate CSAM. They also think it’s essential for third parties like White Circle to ‘highlight where companies’ safeguards are failing.’

White Circle believes that using the RISE method could help frontier labs flag more jailbreaks and rapidly ramp up defenses to block harmful outputs. However, they note that this will only be effective if mandatory external safety testing is implemented.

The public may be stuck with models that simply rewrite bad prompts instead of evolving natural-language attack strategies that elicit LLM’s own exploration capabilities. White Circle thinks governments should push for independent third-party evaluation and monitoring to prevent biases in AI development.