YouTube Auto-Moderation Mistakenly Banned Women of Sex Tech Conference

YouTube’s automated content moderation tool erroneously removed The Women of Sex Tech conference’s live-streamed event and banned the conference from the platform, despite not violating the platform’s sexual content policies.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The automated moderation tool failed to perform reliably, misclassifying educational and compliant content as sexually explicit, demonstrating a lack of robustness in edge-case classification.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: YouTube explicitly increased its reliance on automated algorithms to moderate content without sufficient human oversight, accepting the risk of erroneous bans.
• 1.1 Unfair discrimination and misrepresentation: The automated ban disproportionately affected and silenced sexual expression and sex education, reflecting a broader pattern of censorship targeting these specific communities.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by an erroneous decision made by YouTube's deployed automated moderation algorithm, which unintentionally flagged compliant content as violating guidelines.

EU AI Act risk tier

• Risk tier: 4 Minimal or No Risk

Minimal or No Risk: The automated content moderation tool functions similarly to a spam filter, which is classified as posing minimal or no risk under the EU AI Act, despite its potential to cause erroneous content removals.

AI system and alleged parties

• AI system: YouTube automated content moderation tool (Google)
• AI purpose: Content Moderation; NSFW Content Detection
• Behaviour type: Autonomous
• Alleged developer: YouTube
• Alleged deployer: YouTube
• Alleged harmed parties: Women of Sex Tech conference organizers, Women of Sex Tech conference attendants

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Differential treatment

Reported: Yes, the report describes the incident as part of a pattern of tech companies silencing sex education and sexual speech online.

Directly caused: The AI system erroneously targeted and banned the 'Women of Sex Tech' conference, treating compliant sex education content differently from other compliant streams.

Inferred additional harm: The systematic over-blocking of sex-positive and educational content by automated tools likely leads to broader unequal treatment and marginalization of sex educators and creators.

People affected

• Occurrences reported: 1
• People reportedly harmed: 5
• People reportedly exposed: 5

Potential causes

Management

• Over-Reliance on Automation: Management shifted moderation heavily to AI due to pandemic constraints.
• Acceptance of High False Positives: Management accepted wrongful removals to protect platform ecosystem.

Technology

• Flawed Auto-Moderation Algorithm: The algorithm falsely flagged non-violating content as policy-breaking.
• Lack of Real-Time Context Analysis: The automated tool could not distinguish sex education from pornography.

Data Inputs

• Overly Broad Keyword Triggers: Metadata or titles containing 'sex' likely triggered automated bans.

Human Factors

• Reduced Human Moderator Staffing: Pandemic constraints led to fewer human reviewers to verify flags.

Process and Methods

• Prioritizing Speed Over Accuracy: The automated process favored rapid removal over verifying violations.
• Delayed Appeal Process: Creators had to rely on post-ban appeals rather than pre-ban checks.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, consistent accounts of the incident from multiple sources (Daily Dot, Motherboard/Vice). The role of the AI system (YouTube's auto-moderation algorithm) is explicitly stated by both the platform and the affected parties, and the financial and operational impacts are well-documented.
• Ambiguities identified: The exact financial loss is described generally as 'a couple hundred dollars' rather than a precise figure, and the exact number of team members or registered attendees is not specified.
• Alternative interpretations: None. The incident is unambiguously an erroneous automated content moderation action.

An automated YouTube moderation system erroneously banned a live-stream of the 'Women of Sex Tech' conference due to a false-positive policy violation. While highlighting systemic issues with automated content moderation and minor impacts on digital expression, the incident carries negligible national security implications.

• Overall national security impact: Minor
• Scope: Unknown
• Primary target: Unknown
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The incident represents an ongoing, systemic challenge with automated platform moderation rather than an active national security crisis.
• Autonomy: Full autonomy. The AI system operated independently to scan the live-stream and automatically executed the ban without real-time human intervention or oversight.
• Novelty: Established threat. Erroneous automated content moderation and algorithmic false-positives are well-documented, recurring issues across major social media platforms.

Impact by dimension

• Physical security: Negligible. The incident involved an automated content moderation error on a commercial video-sharing platform, presenting no threat to physical systems, infrastructure, or human safety.
• Information security: Negligible. No information warfare, state-sponsored disinformation campaigns, or intelligence compromises were associated with this unintentional algorithmic error.
• Sovereignty: Negligible. The incident occurred entirely within a private commercial platform and did not impact state authority, electoral processes, or government decision-making.
• Economic security: Negligible. Economic impact was limited to minor alternative hosting costs of a couple hundred dollars, posing no threat to national economic stability or strategic industries.
• Societal stability: Minor. The automated ban represents a minor, localized restriction on freedom of expression and digital rights for a specific community, manageable through standard platform appeal procedures.