Facebook’s and Twitter's Automated Content Moderation Reportedly Failed to Effectively Enforce Violation Rules for Small Language Groups

Facebook's and Twitter were not able to sufficiently moderate content of small language groups such as the Balkan languages using AI, allegedly due to the lack of investment in human moderation and difficulty in AI-solution design for the languages.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.3 Unequal performance across groups

The automated content moderation systems perform with significantly lower accuracy and effectiveness for Balkan language speakers compared to English speakers, leading to unequal protection from hate speech and disproportionate censorship.

Additional risk subdomains

• 1.2 Exposure to toxic content: The failure of the AI to detect hate speech and violent threats in Balkan languages directly exposes users to toxic and harmful content.
• 7.4 Lack of transparency or interpretability: The platforms lack transparency regarding automated moderation decisions, appeal statistics, and the criteria used to filter regional content.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The content moderation failures and wrongful censorship occurred post-deployment due to the AI systems' inability to accurately process Balkan languages, which was an unintentional outcome of the automated moderation process.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The AI content moderation systems pose a moderate risk to freedom of expression and require transparency regarding their automated decisions and appeal processes.

AI system and alleged parties

• AI system: Facebook content moderation AI, Twitter content moderation AI (Meta)
• AI purpose: Content Moderation; Hate Speech Detection
• Behaviour type: Autonomous
• Alleged developer: Twitter, Facebook
• Alleged deployer: Twitter, Facebook
• Alleged harmed parties: Twitter users of small language groups, Facebook users of small language groups

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Substantial
• Differential treatment: direct Minor, indirect Minor
• Civil rights: direct Negligible, indirect Minor
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Negligible, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the spread of hate speech, targeted harassment, and violent threats that the AI systems failed to remove.

Indirectly caused: The AI's failure to effectively moderate Balkan languages allowed hate speech, genocide denial, and violent threats to remain online and continue spreading to a large audience.

Inferred additional harm: It is likely that thousands of toxic posts remained active, exposing a significant portion of the Balkan social media user base to harmful content.

Differential treatment

Reported: The report explicitly describes differential treatment where non-English and small language groups receive lower-quality content moderation and protection.

Directly caused: The automated moderation systems performed poorly for Balkan languages compared to English, resulting in unequal safety standards and higher exposure to hate speech for Balkan users.

Indirectly caused: The lack of investment in human moderation for small language groups led to inconsistent enforcement, where innocuous comments were banned while overt abuse was tolerated.

Inferred additional harm: Linguistic minorities and smaller national groups globally likely experience similar systemic neglect and unequal protection from online harms due to AI moderation biases.

Civil rights

Reported: The report explicitly describes violations of freedom of expression and wrongful censorship, such as the blocking of a Bosnian journalist.

Directly caused: Facebook's automated system blocked Bosnian journalist [PERSON_001] for 24 hours after he posted a historical photo of a detention camp, wrongfully censoring legitimate journalistic content.

Indirectly caused: The inconsistent and automated removal of content without adequate local context amounts to censorship, restricting users' freedom of expression.

Inferred additional harm: Numerous other journalists, activists, and citizens in the Balkans likely had their legitimate posts wrongfully removed or accounts suspended due to automated moderation errors.

Psychological

Reported: The report explicitly mentions that victims of harassment and violent threats experience duress, and content reviewers face serious emotional and PTSD issues.

Indirectly caused: Users who reported targeted harassment and violent threats experienced distress and duress due to the platform's failure to remove the harmful content.

Inferred additional harm: It is highly likely that many users in the Balkans experienced psychological distress, anxiety, or trauma due to unchecked hate speech and harassment, though specific numbers are not quantified.

Epistemic

Reported: The report explicitly describes the spread of genocide denial and hatred, which distorts historical truth.

Indirectly caused: The failure to remove genocide-denying accounts allows false historical narratives to persist and pollute the information ecosystem in the Balkans.

Inferred additional harm: The persistence of false narratives and hate speech likely misleads some users and deepens ethnic divisions by eroding a shared consensus of historical events.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 100

Potential causes

Management

• Profit-Driven Business Model: Platforms rely on inflammatory content to generate ad revenue.
• Underinvestment in Human Review: Firms fail to invest in human moderators for small language groups.
• Lack of Country-Specific Data: Management refuses to disclose moderation data for specific countries.

Technology

• Poor Non-English NLP Performance: AI models perform poorly at detecting hate speech in non-English languages.
• Inability to Grasp Context: AI filters lack nuance, blocking legitimate journalistic posts.
• Non-Romanic Script Limitations: Designing ML solutions for non-Romanic scripts is highly difficult.

Data Inputs

• Lack of Regional Language Data: Small language groups lack data, making AI training difficult.
• Confusing Reporting Inputs: Laborious category selection by users limits clean input data.

Human Factors

• User Language Barriers: Reporting interfaces require English, creating barriers for victims.
• Moderator PTSD and Trauma: Severe emotional trauma limits effective human oversight of violent posts.
• Inconsistent Human Review: Inconsistent decisions tolerate abuse while banning harmless comments.

Process and Methods

• Opaque Appeal Processes: Process lacks transparency regarding processed and reinstated accounts.
• Uniform Rule Application: Applying a single set of rules to diverse global norms is prone to fail.
• Complex Reporting UI: Web-based reporting is confusing and often not in the local language.

Regulatory Environment

• Lack of Political Pressure: No sufficient pressure to force investment in human moderation.
• Weak Government Adaptation: Governments fail to adapt regulations to the needs of the digital age.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides clear, detailed evidence of the systemic failure of automated content moderation in the Balkans, supported by statistical data from surveys, specific examples of wrongful censorship, and expert commentary. The role of the AI systems is explicitly discussed by both platform representatives and digital rights organizations.
• Ambiguities identified: The exact proportion of moderation handled by AI versus human reviewers for Balkan languages is not disclosed by the platforms.
• Alternative interpretations: The failures could be attributed entirely to a lack of corporate investment in human moderators rather than technical limitations of the AI, though the two factors are deeply interconnected.

Automated content moderation failures by major social media platforms in the Balkans have led to the spread of hate speech, violent threats, and genocide denial, alongside wrongful censorship of journalists. This systemic neglect threatens societal stability and human rights in the region by exacerbating ethnic tensions and restricting legitimate speech.

• Overall national security impact: Substantial
• Scope: Multiple nations
• Primary target: Balkans
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. This represents an ongoing, systemic vulnerability in automated moderation rather than an immediate, acute crisis.
• Autonomy: Human-supervised. The AI systems autonomously flag and remove content, but operate within a framework that includes human review and appeal mechanisms, albeit flawed ones.
• Novelty: Established threat. Algorithmic bias and content moderation failures in non-English languages are well-documented, recurring issues for major social media platforms.

Impact by dimension

• Physical security: Negligible. No physical security or critical infrastructure threats were reported in connection with this incident.
• Information security: Minor. The failure to moderate hate speech and genocide denial pollutes the regional information ecosystem, though it does not represent a coordinated foreign intelligence operation.
• Sovereignty: Negligible. No direct threats to state authority, electoral integrity, or core government decision-making processes were identified.
• Economic security: Negligible. The incident involves content moderation failures and does not impact strategic industries, economic stability, or technological competitive advantage.
• Societal stability: Substantial. Systemic failures in moderating Balkan languages allow unchecked hate speech and genocide denial, while wrongfully censoring journalists, threatening social cohesion and human rights in a historically volatile region.