OpenDream AI Platform Reportedly Commercialized AI-Generated CSAM and Non-consensual Deepfake Sexual Images

OpenDream, an AI image generation platform operated by CBM Media Pte Ltd, reportedly allowed users to generate and monetize AI-generated CSAM and non-consensual sexual deepfakes through its public gallery and paid NSFW tools. The content was reported to be publicly accessible and indexed by search engines for months before limited moderation occurred in mid-2024, prompting takedowns by Google and Bing and the termination of its ad and payment services.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.2 Exposure to toxic content

The platform directly exposed the public to toxic and illegal content by hosting and displaying AI-generated child sexual abuse material (CSAM) in an unmoderated public gallery.

Additional risk subdomains

• 4.3 Fraud, scams, and targeted manipulation: The platform was used to generate non-consensual sexual deepfakes of female celebrities, exploiting their public likenesses for user gratification and platform monetization.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The risk was caused by human decisions, specifically the platform operators intentionally offering paid plans for NSFW models and users intentionally prompting the AI to generate CSAM.

EU AI Act risk tier

• Risk tier: 1 Unacceptable

Unacceptable Risk: The system was used to generate child sexual abuse material (CSAM), which is deemed harmful and prohibited outright due to its potential to violate fundamental rights and pose significant dangers to society.

AI system and alleged parties

• AI system: OpenDream
• AI purpose: Image Generation; Deepfake Image Generation
• Behaviour type: Tool
• Alleged developer: CBM Media Pte Ltd
• Alleged deployer: OpenDream users, OpenDream
• Alleged harmed parties: Victims of non-consensual deepfakes, minors, General public, Children

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Minor
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Minor, indirect Minor
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Minor, indirect Minor

Malicious content

Reported: The report explicitly describes the generation and public display of toxic and malicious content, specifically CSAM and non-consensual deepfakes.

Directly caused: The platform directly generated and hosted sexually explicit images of young children, infants, and non-consensual deepfakes of celebrities in its public gallery.

Indirectly caused: The platform's unmoderated content was indexed by Google and Bing, spreading the toxic content to search engine users.

Inferred additional harm: It is likely that thousands of additional toxic images were generated by the platform's 310 paid users and shared across other online forums.

Civil rights

Reported: The report describes violations of human rights, specifically the sexual exploitation of children and the violation of women's rights through non-consensual deepfakes.

Directly caused: The platform directly facilitated the violation of children's rights by generating and publicly hosting synthetic child sexual abuse material.

Indirectly caused: The platform violated the civil rights of female celebrities by generating and hosting non-consensual sexual deepfakes of them.

Inferred additional harm: It is inferred that hundreds of children and women had their fundamental rights to dignity, safety, and bodily integrity violated through the generation of this synthetic content.

Psychological

Reported: The report describes the potential psychological harm of CSAM and non-consensual deepfakes, noting they are 'massively detrimental' and normalize abusive behavior.

Indirectly caused: The creation of non-consensual sexual deepfakes of female web streamers, politicians, singers, and actors causes severe distress and psychological harm to the victims.

Inferred additional harm: It is highly likely that the celebrities and minors depicted in the generated explicit content suffered significant distress, anxiety, and trauma upon discovering the material.

Child sexual exploitation and abuse

Reported: The report explicitly describes a CSEA incident involving the generation and public hosting of AI-generated child sexual abuse material (CSAM).

Directly caused: The platform directly facilitated the generation and public hosting of CSAM, including images of toddlers and infants in sexualized contexts.

Indirectly caused: The platform's lack of moderation allowed these CSAM images to be indexed by major search engines, making them publicly accessible.

Inferred additional harm: It is highly likely that the open-source models used by OpenDream were trained on real sexual imagery of children, thereby indirectly facilitating ongoing child exploitation.

People affected

• Occurrences reported: 1
• People reportedly harmed: 100
• People reportedly exposed: 10000

Potential causes

Management

• Monetizing NSFW Content: Management offered paid tiers specifically for generating NSFW content.
• Neglected Safety Risks: Management failed to implement safety guardrails prior to launching.

Technology

• Open-source NSFW Models: Platform used open-source models with pre-existing NSFW capabilities.
• No Algorithmic Moderation: No automated filters to block illicit prompts or restrict CSAM generation.

Data Inputs

• NSFW Training Data: Base AI models were trained on datasets containing sexualized imagery.
• Illicit User Prompts: Users input explicit terms like 'toddler' combined with sexual actions.

Human Factors

• Malicious User Intent: Users actively sought to generate and share harmful CSAM and deepfakes.

Process and Methods

• Unmoderated Public Gallery: Generated images were automatically posted publicly without prior review.
• Lack of Reporting Tools: No effective mechanism for users to flag or report CSAM on the platform.

Regulatory Environment

• Lagging Global AI Laws: Legal frameworks criminalizing synthetic CSAM are inconsistent globally.
• Weak Infrastructure Policy: Host and registrar failed to proactively enforce anti-CSAM policies.

Information quality

• Classification confidence: High
• Reason for confidence: The report by Bellingcat is highly detailed, providing direct evidence of the platform's operations, monetization, ownership, and the specific types of harmful content generated. It includes screenshots, domain registration details, and statements from search engines and safety organizations.
• Ambiguities identified: The exact number of unique CSAM images generated and the precise number of affected individuals (both minors and celebrities) are not fully quantified.
• Alternative interpretations: None. The evidence of CSAM generation and lack of moderation is clear and unambiguous.

An AI image generation platform facilitated and monetized the creation of synthetic CSAM and non-consensual deepfakes. While representing severe human rights violations and criminal activity, the incident has minor direct national security implications and is primarily a law enforcement and content moderation challenge.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Singapore, United States
• Alleged perpetrator: CBM Media Pte Ltd and platform users

Threat characteristics

• Imminence: Long-term. The platform has been temporarily taken offline and moderated, representing a long-term strategic concern regarding synthetic CSAM rather than an immediate crisis.
• Autonomy: Human-controlled. The AI system functions as a tool, generating images strictly based on direct human prompts and instructions.
• Novelty: Evolved capability. Represents a significant advancement in the ease of generating and monetizing synthetic CSAM and deepfakes, building on established capabilities.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or kinetic targets were affected by this incident.
• Information security: Minor. The platform facilitated non-consensual deepfakes of public figures, but there is no evidence of coordinated state-sponsored information warfare or intelligence compromise.
• Sovereignty: Negligible. No threat to state authority, territorial control, or core government decision-making processes was identified.
• Economic security: Negligible. The financial impact was limited to platform monetization of small sums, with no strategic economic or technological threat.
• Societal stability: Minor. While the generation of CSAM and non-consensual deepfakes represents severe human rights violations, the national security impact on societal stability remains minor and manageable via standard law enforcement.