Microsoft’s Algorithm Allegedly Selected Photo of the Wrong Mixed-Race Person Featured in a News Story

A news story published on MSN.com featured a photo of the wrong mixed-race person that was allegedly selected by an algorithm, following Microsoft’s layoff and replacement of journalists and editorial workers at its organizations with AI systems.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.1 Unfair discrimination and misrepresentation

The AI system incorrectly paired a story about racism with a photo of the wrong mixed-race bandmate, resulting in the misrepresentation of women of color on a highly visible platform.

Additional risk subdomains

• 6.2 Increased inequality and decline in employment quality: Microsoft laid off dozens of professional journalists and editors to replace their roles with automated AI curation software.
• 7.3 Lack of capability or robustness: The AI system failed to perform reliably, pairing an incorrect image with the article headline due to a failure in its experimental image-selection feature.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The misidentification error was caused by the AI system's automated image selection feature after it was deployed to curate news on MSN.com.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system is an automated content curation and presentation tool on a public news platform, which falls under transparency obligations for AI-generated or curated content.

AI system and alleged parties

• AI system: Microsoft automated news curation system (Microsoft)
• AI purpose: Automated Content Curation; Content Recommendation
• Behaviour type: Autonomous
• Alleged developer: Microsoft
• Alleged deployer: MSN.com, Microsoft
• Alleged harmed parties: Leigh-Anne Pinnock, Jade Thirlwall

Harm severity

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Minor
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: The report explicitly describes financial harm in the form of job losses for dozens of journalists, though specific monetary values are not detailed.

Directly caused: N/A

Indirectly caused: Approximately 77 contracted journalists and editors in the US and UK lost their jobs and regular income due to Microsoft's decision to automate their roles.

Inferred additional harm: N/A

Differential treatment

Reported: The report explicitly describes differential treatment and misrepresentation of mixed-race individuals.

Directly caused: The AI system failed to differentiate between two women of color, pairing a story about Jade Thirlwall's experience with racism with a photo of Leigh-Anne Pinnock.

Indirectly caused: N/A

Inferred additional harm: N/A

Epistemic

Reported: The report explicitly describes epistemic harm in the form of publishing false and misleading information on a major news platform.

Directly caused: The AI system published a misleading news presentation on MSN.com by pairing the wrong image with the headline, misrepresenting the subject of the story to millions of readers.

Indirectly caused: N/A

Inferred additional harm: N/A

People affected

• Occurrences reported: 1
• People reportedly harmed: 79
• People reportedly exposed: 1000000

Potential causes

Management

• Cost-Cutting Staff Layoffs: Replaced experienced editorial staff with AI to reduce operational costs.
• Rapid Transition to Automation: Abrupt termination of human contracts created immediate gaps in oversight.
• Prioritizing AI over Reputation: Shifted to automated curation despite known risks of publishing errors.

Technology

• Experimental Image Selection Feature: Feature paired an alternate image from the page with the headline incorrectly.
• Automated Curation Limitations: AI could not contextualize or verify image-headline pairings before publishing.
• Feedback Loop in AI Curation: AI repeatedly selected articles criticizing itself to publish on the homepage.

Data Inputs

• Multiple Images on Source Page: Source articles contained multiple images, confusing the automated selector.
• Lack of Metadata Validation: System did not validate if selected alternate images matched the headline.

Human Factors

• Complete Reliance on Automation: Microsoft replaced human editors entirely, removing manual verification steps.
• Overruling of Human Interventions: AI system could overrule remaining staff attempts to delete critical articles.

Process and Methods

• Live Testing of Experimental Features: Microsoft tested the alternate image selection feature directly in production.
• Insufficient Pre-Deployment Testing: Experimental features were deployed without adequate safety checks or guards.
• Lack of Automated Quality Control: No automated checks to prevent mismatched headlines and images from going live.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts of the layoffs, the specific error, the public reactions, and Microsoft's technical explanation of the image-pairing bug.
• Ambiguities identified: While the public and the affected individuals perceived the error as racial bias, Microsoft claimed it was a purely technical bug in an experimental image-selection feature.
• Alternative interpretations: The incident could be interpreted as a standard software integration bug rather than an AI-specific safety or bias failure.

An automated news curation system deployed by Microsoft mistakenly paired an article about racism with a photo of the wrong band member of color. While the incident highlighted issues of algorithmic bias, automated editorial risks, and corporate job displacement, it has negligible implications for national security.

• Overall national security impact: Negligible
• Response level: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The incident represents a resolved corporate event with no active or imminent national security threat.
• Autonomy: Full autonomy. The AI system selected and published the content autonomously on MSN.com without prior human review or intervention.
• Novelty: Established threat. Algorithmic misidentification and automated curation errors are well-documented, established technical issues.

Impact by dimension

• Physical security: Negligible. No physical security, critical infrastructure, or kinetic threats were involved in this commercial AI deployment incident.
• Information security: Negligible. The incident involved an automated editorial error on a commercial news platform, not state-sponsored disinformation or intelligence compromise.
• Sovereignty: Negligible. No core government functions, electoral systems, or sovereign state authorities were impacted by this commercial automation change.
• Economic security: Negligible. While resulting in localized job losses for contracted editors, the incident posed no threat to strategic industries or national economic security.
• Societal stability: Negligible. The incident caused localized psychological distress and highlighted algorithmic bias, but did not threaten large-scale social cohesion or civil liberties.