Biased Google Image Results

On June 6, 2016, Google image searches of "three black teenagers" resulted in mostly mugshot images whereas Google image searchers of "three white teenagers" consisted of mostly stock images, suggesting a racial bias in Google's algorithm.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.1 Unfair discrimination and misrepresentation

The search engine's output unfairly represented and stereotyped Black teenagers by predominantly associating them with mugshots and criminality, leading to biased representation.

Additional risk subdomains

• 1.3 Unequal performance across groups: The search engine's effectiveness and dignity of output differed significantly based on the demographic group specified in the query.
• 7.3 Lack of capability or robustness: The algorithm failed to robustly handle sensitive demographic queries, lacking the capability to account for social context or systemic bias.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The biased search results were generated by Google's deployed search algorithm, which unintentionally reflected and amplified societal prejudices present in web data.

EU AI Act risk tier

• Risk tier: 4 Minimal or No Risk

Minimal or No Risk: General-purpose search engines and simple information retrieval tools are generally classified as posing minimal or no risk under the EU AI Act, as they do not fall under prohibited or high-risk categories.

AI system and alleged parties

• AI system: Google Image Search (Google)
• AI purpose: Image Search; Content Recommendation
• Behaviour type: Tool
• Alleged developer: Google
• Alleged deployer: Google
• Alleged harmed parties: Minority Groups

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Minor, indirect Minor
• Civil rights: direct Negligible, indirect Minor
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Minor, indirect Minor
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes, the reports describe the search engine spreading and prioritizing negative, stereotyping images of Black teenagers.

Directly caused: The Google Search algorithm surfaced and prioritized mugshots and negative images for the query 'three black teenagers', exposing users to biased and potentially harmful content.

Differential treatment

Reported: Yes, the reports explicitly describe the starkly different search results returned for Black versus white teenagers.

Directly caused: The search algorithm treated queries differently based on race, representing Black teenagers as criminals (mugshots) and white teenagers as wholesome (stock photos).

Indirectly caused: The search results reinforced societal biases and racial profiling by presenting a skewed representation of Black youth.

Inferred additional harm: We infer that similar differential treatment occurred across other demographic queries, such as those related to hairstyles or other minority groups.

Civil rights

Reported: Yes, academic Safiya Noble is cited arguing that automated decision-making systems are becoming a human rights issue due to systemic bias.

Indirectly caused: The perpetuation of the 'dangerous Black youth' stereotype can indirectly contribute to systemic civil rights issues, such as biased policing, harsher sentencing, or employment discrimination.

Inferred additional harm: We infer that the systemic reinforcement of racial bias by dominant search engines indirectly undermines the civil rights and equal treatment of Black individuals in society.

Psychological

Reported: Yes, the reports describe users being shocked, disturbed, and angered by the racially biased search results.

Directly caused: Kabir Alli and thousands of social media users experienced shock, distress, and indignation upon seeing the stark racial disparities in the search results.

Indirectly caused: The reinforcement of negative racial stereotypes can cause long-term psychological harm, feelings of marginalization, and diminished self-worth among Black youth.

Inferred additional harm: We infer that thousands of Black youth and other users experienced minor psychological distress, alienation, or a sense of systemic exclusion due to these search results.

Epistemic

Reported: Yes, the reports discuss how search engines shape public perception and what users believe to be the truth.

Directly caused: The algorithm presented a highly skewed, unrepresentative set of mugshots as the primary reality for 'three black teenagers', misleading users about the actual demographic distribution of criminality.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 60000

Potential causes

Management

• Prioritization of Fiduciary Interests: Google prioritizes business interests over active search result curation.
• Deflection of Responsibility: Management blames societal bias rather than taking accountability.
• Inadequate Risk Assessment: Google failed to assess risks of racial stereotyping in search queries.

Technology

• Algorithmic Popularity Bias: The search algorithm prioritizes highly clicked and shared negative images.
• Relying on Alt-Tags and Metadata: The technology relies on human-created alt-tags which can be biased.
• No Algorithmic Context Awareness: The algorithm cannot understand social context or racial implications.

Data Inputs

• Biased Media Representation: News sites disproportionately associate black teens with crime stories.
• Lack of Positive Stock Photos: Fewer stock photos of black teens exist due to market demographic demand.
• Unbalanced Arrest Coverage: Black arrests receive disproportionately high media coverage and tagging.

Human Factors

• Societal Prejudice and Bias: User search and clicking behaviors reflect institutional racial prejudices.
• Media Stereotyping: Writers select mugshots for black teens but wholesome photos for white ones.
• Overreliance on Search Credibility: Users trust search results as objective truths rather than biased reflections.

Process and Methods

• Unvetted Automated Indexing: Web crawling relies on unverified user-generated metadata and captions.
• No Proactive Query Filtering: Google lacks processes to proactively filter or balance sensitive search terms.
• Lack of Transparent Output Vetting: Search results are displayed without transparency of how they are manipulated.

Regulatory Environment

• Lack of Algorithmic Bias Regulation: No legal frameworks exist to hold search engines accountable for bias.
• Narrow Definition of Consumer Harm: Regulators fail to recognize representational bias as actionable harm.
• Lack of Oversight on Tech Monopolies: Regulatory bodies lack oversight on search engine indexing practices.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts of the viral incident, Google's response, and the underlying algorithmic mechanisms (SEO, metadata, search frequency). Multiple sources confirm the exact search terms and results, leaving little ambiguity about what occurred.
• Ambiguities identified: None of significance; the technical explanation of how Google's algorithm ranks images is widely agreed upon across the reports.
• Alternative interpretations: Some commentators argued the issue is purely a reflection of societal bias rather than an AI failure, but this does not change the classification of the AI's role in reproducing that bias.

In June 2016, a viral Google Image search disparity highlighted systemic racial bias in search algorithms, returning mugshots for 'three black teenagers' and positive stock photos for 'three white teenagers'. While sparking significant public debate on algorithmic discrimination, the incident posed minor national security risks, primarily serving as an early public example of unintentional bias.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Algorithmic bias in commercial search engines represents an ongoing, long-term societal and strategic concern rather than an immediate crisis.
• Autonomy: Human-supervised. The search engine indexes and serves results autonomously, though developers maintain the capability to manually intervene and adjust algorithms.
• Novelty: Evolved capability. While search algorithms are established, this incident served as a prominent early example of public exposure to systemic algorithmic bias.

Impact by dimension

• Physical security: Negligible. No physical infrastructure, kinetic assets, or human safety systems were impacted or targeted in this incident.
• Information security: Negligible. There was no evidence of coordinated foreign disinformation, espionage, or compromise of intelligence capabilities.
• Sovereignty: Negligible. The incident did not disrupt government operations, electoral processes, or state authority.
• Economic security: Negligible. No critical economic infrastructure, financial systems, or strategic technologies were compromised.
• Societal stability: Minor. The incident exposed systemic bias in a major information platform, sparking public debate on racial representation, but did not threaten national social cohesion or cause civil unrest.