Facebook's AI Put "Primates" Label on Video Featuring Black Men

Facebook's AI mislabeled video featuring Black men as a video about "primates," resulting in an offensive prompt message for users who watched the video.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.1 Unfair discrimination and misrepresentation

The AI system mislabeled Black men as 'primates', representing them in a highly offensive, historically racist, and discriminatory manner.

Additional risk subdomains

• 1.3 Unequal performance across groups: The report highlights that computer vision and facial recognition technologies perform with lower accuracy and higher error rates for people of color.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The offensive prompt was generated by an active, deployed recommendation AI system due to an unintended classification error.

EU AI Act risk tier

• Risk tier: 4 Minimal or No Risk

Minimal or No Risk: The AI system is a video recommendation and content tailoring feature used on a social media platform, which falls under simple applications with low direct risk to safety or critical infrastructure.

AI system and alleged parties

• AI system: Facebook's AI (Meta)
• AI purpose: Content Recommendation; Image Classification
• Behaviour type: Autonomous
• Alleged developer: Facebook
• Alleged deployer: Facebook
• Alleged harmed parties: Facebook users, Black people

Harm severity

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes, the AI generated and spread an offensive recommendation prompt.

Directly caused: The system generated a prompt associating Black men with 'primates', exposing users to offensive content.

Indirectly caused: N/A

Inferred additional harm: N/A

Differential treatment

Reported: Yes, the AI system subjected Black men to offensive misrepresentation.

Directly caused: Black men in the video were algorithmically associated with 'primates', a deeply offensive racial trope.

Indirectly caused: N/A

Inferred additional harm: This reinforces systemic racial biases and stereotypes against Black people at scale.

People affected

• Occurrences reported: 1
• People reportedly harmed: 2
• People reportedly exposed: 2

Potential causes

Management

• Deprioritization of Racial Risks: Leadership did not prioritize safety oversight for racial bias issues.

Technology

• Flawed Recommendation Engine: The system automatically pushed offensive content tags to users.
• Biased Image Recognition: The underlying computer vision model misidentified humans as primates.

Data Inputs

• Biased Image Repositories: Training datasets lacked diverse and balanced representation of Black people.

Human Factors

• Overreliance on Automation: Engineers trusted automated prompts without active human moderation.

Process and Methods

• Inadequate Bias Testing: Recommendation features were deployed without robust racial bias audits.

Information quality

• Classification confidence: High
• Reason for confidence: The incident is clearly described by a reputable source (The New York Times) with direct statements from Facebook admitting the error and explaining the action taken to disable the feature.
• Ambiguities identified: The exact number of users who saw the prompt is not specified, and the precise technical root cause of the misclassification is not detailed.

Facebook's recommendation AI mistakenly labeled a video featuring Black men with a 'primates' prompt. While highlighting persistent and concerning racial bias in commercial computer vision models, the incident was unintentional and poses negligible directly-attributable threats to national security, representing a minor societal impact manageable through standard corporate and regulatory procedures.

• Overall national security impact: Minor
• Response level: Moderate
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: United States, United Kingdom
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding algorithmic bias rather than an active, imminent national security crisis.
• Autonomy: Full autonomy. The recommendation system automatically analyzed video content and generated the prompt without human intervention.
• Novelty: Established threat. Similar incidents of computer vision systems mislabeling Black individuals have occurred previously, such as the Google Photos incident in 2015.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or human safety capabilities were threatened or impacted by this classification error.
• Information security: Negligible. The incident was an unintentional algorithmic misclassification and did not involve coordinated information warfare, deepfake operations, or intelligence compromise.
• Sovereignty: Negligible. No state authority, electoral processes, or government decision-making systems were disrupted or threatened.
• Economic security: Negligible. While highlighting technical limitations in computer vision, the incident did not threaten strategic industries, economic stability, or critical supply chains.
• Societal stability: Minor. The incident caused public offense and highlighted racial bias in commercial AI, but did not trigger large-scale civil unrest or systematic state-sponsored algorithmic oppression.