AI-Generated Fake News Targets Black Celebrities on YouTube

YouTube faced a surge of AI-generated fake news targeting Black celebrities, including fake narratives about Sean “Diddy” Combs and others. These videos, blending AI-generated and manipulated media, amassed millions of views, challenging content moderation efforts and highlighting the spread of disinformation.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves the intentional creation of humiliating and sexualized synthetic content targeting specific Black celebrities for personal monetization and views.

Additional risk subdomains

• 3.2 Pollution of information ecosystem and loss of consensus reality: The flood of highly engaging AI-generated fake news videos threatens the broader information ecosystem and degrades shared reality.
• 1.2 Exposure to toxic content: The videos expose platform users to unwanted sexualization and harassment through synthetic media depicting lewd acts.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The risk is caused by the intentional decisions and actions of human creators who use AI tools to generate and spread fake news for profit.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes AI-generated content such as deepfakes, which require specific transparency obligations to ensure users are informed of the AI's nature.

AI system and alleged parties

• AI system: AI Speech Classifier model, Reality Defender, unspecified (Reality Defender)
• AI purpose: Voice Generation; Deepfake Image Generation
• Behaviour type: Tool
• Alleged developer: Unknown generative AI tools creators, Unknown AI text-to-speech technology developers
• Alleged deployer: Variety of YouTube content creators
• Alleged harmed parties: Steve Harvey, Sean “Diddy” Combs, General public, Denzel Washington, Black celebrities, Bishop T.D. Jakes

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Severe. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Substantial, indirect Negligible
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Severe, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes videos containing targeted harassment, unwanted sexualization, and lewd acts.

Directly caused: Widespread dissemination of toxic deepfake videos and thumbnails depicting celebrities in lewd acts across YouTube, reaching millions of viewers.

Inferred additional harm: Additional toxic content generated on other platforms like TikTok as mentioned by Nwandu.

Differential treatment

Reported: The report explicitly describes systematic targeting of Black celebrities and exploitation of cultural trust.

Directly caused: Dozens of Black celebrities were disproportionately targeted with salacious fake narratives.

Psychological

Reported: The report explicitly describes targeted harassment and unwanted sexualization of celebrities, which constitutes psychological harm.

Directly caused: Celebrities like T.D. Jakes and Sean Combs suffered public distress and reputational damage, forcing Jakes to address the lies in a sermon.

Inferred additional harm: Widespread distress and anxiety among the dozens of targeted celebrities and their families due to highly realistic deepfakes depicting them in lewd acts.

Epistemic

Reported: The report explicitly describes the flood of AI-generated fake news and salacious disinformation.

Directly caused: Millions of viewers were exposed to fabricated news stories (e.g., Diddy being arrested on Jan 14), leading to confusion and audience members demanding coverage of fake stories.

Inferred additional harm: Erosion of shared reality and trust in legitimate news sources among the public.

People affected

• Occurrences reported: 1
• People reportedly harmed: 24
• People reportedly exposed: 21000000

Potential causes

Management

• Engagement Prioritization: Platform management prioritized metrics that favored sensational fake news.
• Inadequate Proactive Safeguards: Management failed to deploy automated preventative filters for deepfakes.

Technology

• Generative AI Voice Tools: Allowed easy creation of highly realistic fake celebrity voiceovers.
• AI Recommendation Algorithms: Prioritized high-engagement fake news, pushing it to top search results.
• Automated Thumbnail Generation: AI tools enabled rapid creation of misleading and manipulated thumbnails.

Data Inputs

• Unlabeled Synthetic Media: Lack of metadata or labels made it hard to flag AI content automatically.
• Remixing Real and Fake Data: Blending real news events with AI falsehoods deceived detection systems.
• Manipulated Media Inputs: Using real celebrity photos mixed with AI-generated elements to mislead.

Human Factors

• Audience Trust in Alternative Media: Distrust in mainstream media led viewers to believe fake community channels.
• Financial Motivations of Creators: Freelancers and creators exploited AI to churn out high-volume clickbait.
• Difficulty in Detecting Deepfakes: Average viewers struggled to distinguish synthetic audio from real voices.

Process and Methods

• Delayed Policy Implementation: The announced synthetic media labeling policy was not yet active.
• Slow Content Moderation Response: Channels flourished for long periods before YouTube took enforcement action.
• Flawed Monetization Review: Channels profited from ads on fake news before being demonetized.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides detailed, credible information about the channels, views, and specific celebrities targeted, backed by expert analysis from the University of Buffalo and Reality Defender.
• Ambiguities identified: It can be difficult to distinguish between AI-generated media and media that is manipulated without the use of AI (cheapfakes).
• Alternative interpretations: Some channels claimed they only used AI for text-to-speech and not for scripts or thumbnails, suggesting a hybrid of manual and AI manipulation.

An influx of AI-generated disinformation and synthetic media targeted Black celebrities on YouTube to exploit cultural trust and generate ad revenue. While demonstrating the ease of scaling highly engaging fake content using generative AI, the incident remains a commercial fraud and content moderation issue rather than a strategic national security threat.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing trend of generative AI exploitation for financial gain rather than an imminent national security crisis.
• Autonomy: Human-controlled. AI systems were utilized as tools by human actors who directed the generation of specific synthetic voices and images.
• Novelty: Evolved capability. Represents an evolution of existing celebrity clickbait and disinformation tactics, scaled and accelerated through accessible generative AI tools.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or human safety elements were impacted or threatened by this incident.
• Information security: Minor. Involves AI-generated disinformation at scale with millions of views, but targets celebrities for financial gain rather than state-sponsored strategic geopolitical operations.
• Sovereignty: Negligible. No government functions, electoral systems, or state authority operations were targeted or compromised.
• Economic security: Negligible. Impact is limited to platform monetization exploitation and individual celebrity reputation, with no threat to strategic industries or national economic stability.
• Societal stability: Minor. Exploits cultural trust within specific demographic groups and contributes to epistemic degradation, but does not represent a large-scale threat to national social cohesion.