"The Little Mermaid" Clip Doctored Using Generative AI to Replace Black Actress with White Character

A Twitter user reportedly modified using generative AI a short clip of Disney's 2022 version of "The Little Mermaid," replacing a Black actress with a white digital character.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.2 Exposure to toxic content

The incident involved the creation and viral dissemination of racially offensive synthetic media ('whitewashing' a Black actress), exposing social media users to toxic and discriminatory content.

Additional risk subdomains

• 1.1 Unfair discrimination and misrepresentation: The AI tool was used to misrepresent the race of a Black actress, reinforcing discriminatory attitudes and unfair representation.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was caused by a human user intentionally using deployed generative AI tools to modify the trailer and replace a Black actress with a white character.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system was used to generate AI-altered content (a deepfake or synthetic media), which falls under transparency obligations for AI-generated content under the EU AI Act.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Deepfake Video Generation; Image Retouching
• Behaviour type: Tool
• Alleged developer: unknown
• Alleged deployer: @TenGazillioinIQ
• Alleged harmed parties: Halle Bailey, Black actresses

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Minor
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the creation and viral spread of a racially offensive, 'whitewashed' version of 'The Little Mermaid' trailer.

Directly caused: An AI scientist used generative AI to replace the Black lead actress with a white digital character, creating a racially offensive video that was shared on Twitter.

Indirectly caused: The viral tweet led to widespread online arguments, racist commentary, and the subsequent suspension of the accounts involved.

Differential treatment

Reported: The report describes the digital replacement of a Black actress with a white character, which critics labeled as racist 'whitewashing'.

Directly caused: The AI tool was used to selectively target and alter the race of the Black lead actress while keeping her singing voice, representing discriminatory treatment of her likeness.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 100000

Potential causes

Management

• Absence of AI Developer Oversight: No institutional oversight to prevent researchers from creating biased AI.
• Inadequate Platform Moderation: Social media platforms struggled to proactively block abusive AI content.

Technology

• Advanced Deepfake Capabilities: AI models allow rapid, realistic alteration of human faces and skin tones.
• High-Performance Compute Access: Access to powerful GPUs enabled rendering the modified trailer in 24 hours.

Data Inputs

• Unsanctioned Video Data Use: Using copyrighted trailer footage without permission to run AI modification.
• Targeted Demographic Modification: Inputting race-based parameters to alter the main character's appearance.

Human Factors

• Racial Bias and Backlash: Users sought to alter the black actress's race due to racist motivations.
• Lack of Ethical Awareness: Developer claimed purely educational intent, ignoring the offensive impact.

Process and Methods

• Rapid AI Video Generation: Lack of ethical checkpoints during the rapid video modification process.
• Viral Social Media Distribution: Sharing controversial AI modifications on Twitter to generate engagement.

Regulatory Environment

• Reactive Platform Moderation: Twitter suspended accounts only after the offensive content went viral.
• Lack of Deepfake Regulations: Absence of clear legal frameworks governing unauthorized AI whitewashing.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly details the incident, the motivation of the users, the reaction of the platform (suspension), and the nature of the AI-generated content. There is little ambiguity about what transpired.
• Ambiguities identified: The specific AI model or software used by the scientist is not named, referred to only as 'Artificial Intelligence' running on '4x A6000'.

An individual used generative AI to racially alter a movie trailer, replacing a Black actress with a white character. While the incident generated significant online controversy and highlighted the risks of discriminatory synthetic media, it carried negligible national security implications.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: An individual Twitter user

Threat characteristics

• Imminence: Long-term. The incident is a past event with no ongoing immediate or near-term national security crisis.
• Autonomy: Human-controlled. The AI tool was directly controlled and utilized by a human operator to perform the video modification.
• Novelty: Established threat. The use of generative AI and deepfake technology to alter video content and likenesses is an established threat.

Impact by dimension

• Physical security: Negligible. No physical security, critical infrastructure, or kinetic threats were involved in this incident.
• Information security: Negligible. While synthetic media was created, it was a movie trailer modification by an individual, not a state-sponsored information warfare campaign targeting national institutions.
• Sovereignty: Negligible. No sovereign territory, electoral systems, or government decision-making processes were affected by this incident.
• Economic security: Negligible. No financial systems, strategic technology theft, or critical supply chains were impacted.
• Societal stability: Negligible. Although the incident involved racially offensive synthetic media causing online controversy, it did not threaten national-scale societal stability or civil liberties.