AI Photo Filter Lightens Skin, Changes Eye Color in Student's 'Professional' Image

An AI application modified an MIT student's photo to appear 'professional' by lightening her skin and changing her eye color to blue, highlighting the racial bias in the training data of the program.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.1 Unfair discrimination and misrepresentation

The AI system misrepresented the Asian-American user by altering her racial features to appear Caucasian in response to a request for a 'professional' photo, reflecting underlying racial bias.

Additional risk subdomains

• 7.3 Lack of capability or robustness: The system failed to reliably preserve the user's facial identity and robustly handle diverse demographic inputs during image restyling.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The biased image generation was an unexpected and unintentional outcome produced by the deployed Playground AI model when processing a user's prompt.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: The system is an AI image generator producing synthetic content, which falls under the category of AI-generated content requiring transparency obligations.

AI system and alleged parties

• AI system: Playground AI
• AI purpose: Image Generation; Image Retouching
• Behaviour type: Tool
• Alleged developer: Playground AI
• Alleged deployer: Playground AI
• Alleged harmed parties: Rona Wang, Racial minorities who may have experienced the same result

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 1

Potential causes

Management

• Deploying Unsuited Tools: Releasing tools to public without restricting unsupported use cases.

Technology

• Lack of Identity Preservation: Model cannot preserve face identity and restylize it properly.
• Uninstructable Model Design: Model picks generic features based on prompts and is not smart enough.

Data Inputs

• Biased Training Data: Training photos mostly depict white people in professional scenes.
• Underrepresentation of Minorities: Data troves do not accurately reflect various racial and ethnic groups.

Human Factors

• Unsupported Use Case Attempt: User tried using the tool for photo avatars which is not supported.
• Overreliance on AI Tools: Users put too much trust in AI and assume it is unbiased.

Process and Methods

• Insufficient Bias Mitigation: Lack of processes to detect and mitigate racial bias in outputs.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly details the user's experience, the specific AI tool used, the exact input and output, and includes direct responses from the startup's founder. There is little ambiguity about what transpired.
• Ambiguities identified: The technical reason for the bias is speculated (training data bias vs random generation) but the event itself is clear.
• Alternative interpretations: The founder suggested the result could be a random 'dice roll' rather than systematic bias, though experts point to underlying training data bias.

An MIT student experienced racial bias when using Playground AI to generate a professional profile picture, resulting in a white-washed image. While the incident generated significant public discussion regarding AI bias and representation, it is a localized consumer software issue with negligible national security implications.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Reflects ongoing strategic and ethical discussions around AI bias rather than an active national security crisis.
• Autonomy: Human-controlled. The AI tool acted as an assistant generating an image based directly on user-provided photo and text prompts.
• Novelty: Established threat. Racial and demographic biases in generative image models are well-documented phenomena with numerous prior public examples.

Impact by dimension

• Physical security: Negligible. The incident involves a consumer image generation tool and poses no threat to physical systems, critical infrastructure, or human safety.
• Information security: Negligible. No classified information, intelligence capabilities, or systematic state-sponsored information operations were involved or compromised.
• Sovereignty: Negligible. The event does not impact state authority, government decision-making, electoral systems, or constitutional processes.
• Economic security: Negligible. While highlighting commercial software limitations, the incident does not threaten financial stability, strategic industries, or national technological advantage.
• Societal stability: Negligible. The incident sparked public discussion about AI bias, but represents an isolated consumer experience with negligible impact on broad societal stability or civil liberties.