Gender Biases of Google Image Search

Google Image returns results that under-represent women in leadership roles, notably with the first photo of a female "CEO" being a Barbie doll after 11 rows of male CEOs.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.1 Unfair discrimination and misrepresentation

The incident involves the systematic underrepresentation and stereotyping of women in search results, leading to unfair representation of this group.

Additional risk subdomains

• 3.1 False or misleading information: The biased search results misled users, shifting their real-world perceptions of occupational gender ratios by 7%.
• 6.2 Increased inequality and decline in employment quality: The ad delivery system showed significantly fewer high-paying executive job ads to women, potentially restricting their employment opportunities.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The bias was caused by Google's search and ad algorithms processing data post-deployment, resulting in unintentional discriminatory outputs.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The report describes an AI system used in employment and recruitment advertising, which falls under systems affecting access to employment and vocational opportunities.

AI system and alleged parties

• AI system: Google Image Search (Google)
• AI purpose: Image Search; Ad Delivery
• Behaviour type: Tool
• Alleged developer: Google
• Alleged deployer: Google
• Alleged harmed parties: Women

Harm severity

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Differential treatment

Reported: The report explicitly describes systematic differential treatment of women in both image search results and job advertisement delivery.

Directly caused: Google's ad system showed high-paying executive job ads 1,852 times to male profiles but only 318 times to identical female profiles. Additionally, women were significantly underrepresented in search results for leadership roles like 'CEO'.

Indirectly caused: N/A

Inferred additional harm: Millions of female job seekers and search engine users were likely subjected to biased information and restricted career opportunities due to these algorithmic disparities.

Civil rights

Reported: The report explicitly describes potential violations of civil rights, specifically gender discrimination in employment advertising and representation.

Directly caused: The ad-delivery system restricted women's access to high-paying job advertisements, which Carnegie Mellon researchers identified as emerging discrimination in the ad ecosystem.

Indirectly caused: N/A

Inferred additional harm: Widespread algorithmic bias in search and advertising platforms may systematically undermine equal opportunity and civil rights for women on a global scale.

Epistemic

Reported: The report explicitly describes epistemic harm, where biased search results altered users' perceptions of real-world gender ratios.

Directly caused: Exposure to skewed Google Image Search results shifted study participants' estimates of occupational gender ratios by an average of 7 percent.

Indirectly caused: N/A

Inferred additional harm: Continuous exposure to biased search results likely reinforces and perpetuates societal gender stereotypes and false beliefs about professional capabilities at a massive scale.

People affected

• Occurrences reported: 2
• People reportedly harmed: 1000
• People reportedly exposed: 1000

Potential causes

Management

• Prioritizing User Preferences: Designing algorithms to show what they think users want to see.
• Lack of Accountability: Search companies declining to comment or address systemic bias actively.

Technology

• Algorithmic Bias: Algorithms reflect and exaggerate societal stereotypes in search results.
• Ad-Personalization Bias: Ad delivery system shows fewer high-paying executive ads to women.
• Image Recognition Flaws: Flickr auto-tagging tool misidentified Black people as animals.

Data Inputs

• Biased Web Content: Search results reflect biased naming, labeling, and linking on the web.
• Stereotypical Stock Photos: Media and stock libraries underrepresent diverse professionals.
• Skewed Training Data: Algorithms learn from historically biased search histories and clicks.

Human Factors

• User Search Behavior: Sexist and racist search habits reinforce biased algorithmic learning.
• Societal Gender Stereotypes: Users hold existing preconceptions about gender ratios in professions.
• Advertiser Targeting Choices: Advertisers select specific demographics, excluding women from executive ads.

Process and Methods

• Lack of Algorithmic Transparency: Algorithms operate in closed boxes without external scrutiny.
• Engagement-Based Ranking: Algorithms prioritize PageRank and click-through rates over fairness.
• Inadequate Search Auditing: Lack of systematic methods to evaluate and balance search representation.

Information quality

• Classification confidence: High
• Reason for confidence: The assessment is based on two highly credible, peer-reviewed academic studies from major universities (University of Washington/Maryland and Carnegie Mellon University) with specific, quantified data points. The reports consistently describe the same core findings regarding search bias and ad delivery disparities.
• Ambiguities identified: The exact relative contribution of advertiser targeting choices versus Google's internal algorithmic optimization in the CMU ad study remains somewhat ambiguous.
• Alternative interpretations: The search results could be interpreted as merely reflecting existing web content biases rather than an algorithmic failure, though the studies show the algorithm actively exaggerates these biases.

Studies in 2015 revealed systemic gender bias in Google's Image Search and ad-delivery algorithms, showing underrepresentation of women in leadership roles and discriminatory ad targeting for high-paying jobs. While this represents a notable societal concern regarding algorithmic fairness and discrimination, its national security impact is negligible to minor, lacking any physical threat, state-sponsored information warfare, or compromise of sovereign functions.

• Overall national security impact: Minor
• Response level: Moderate
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Multiple nations
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing, systemic algorithmic design and data bias issue rather than an active crisis.
• Autonomy: Human-controlled. The systems function as tools that process queries and deliver ads based on pre-defined algorithms without independent decision-making.
• Novelty: Evolved capability. Demonstrated a significant advancement in quantifying how algorithmic systems scale and reinforce societal biases in digital ecosystems.

Impact by dimension

• Physical security: Negligible. No physical security threat or impact on critical infrastructure was reported.
• Information security: Negligible. The incident represents unintentional algorithmic bias rather than a coordinated information warfare campaign or intelligence compromise.
• Sovereignty: Negligible. No threat to state sovereignty, border control, or core government operations was identified.
• Economic security: Negligible. The bias affected commercial ad delivery and image searches, with no direct threat to strategic industries or national economic security.
• Societal stability: Minor. The algorithmic bias represents systematic gender discrimination and shifts in public perception at scale, affecting civil rights but manageable through standard regulatory and corporate policy adjustments.