Sexist and Racist Google Adsense Advertisements

Advertisements chosen by Google Adsense are reported as producing sexist and racist results.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.1 Unfair discrimination and misrepresentation

The AI system disproportionately associated black-identifying names with arrest records, resulting in unfair representation and racial discrimination.

Additional risk subdomains

• 7.4 Lack of transparency or interpretability: The 'black box' nature of Google's AdSense algorithm made it difficult for researchers to pinpoint the exact cause of the discriminatory ad delivery.
• 6.2 Increased inequality and decline in employment quality: The discriminatory ads risked harming the employment prospects of black job seekers whose names were searched by potential employers.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The discriminatory ad delivery was an unexpected, unintentional outcome of Google's automated ad-serving algorithms operating in a post-deployment environment.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The system's outputs directly impact employment and worker management, specifically affecting access to employment and recruitment, which is classified as high risk under the EU AI Act.

AI system and alleged parties

• AI system: Google AdSense (Google)
• AI purpose: Ad Delivery; Behavioral Modeling
• Behaviour type: Autonomous
• Alleged developer: Google
• Alleged deployer: Google
• Alleged harmed parties: Women, Minority Groups

Harm severity

Highest direct severity in any category: Severe. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Substantial, indirect Negligible
• Civil rights: direct Substantial, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Differential treatment

Reported: The report explicitly describes systemic differential treatment where black-identifying names were 25 percent more likely to receive arrest-related ads.

Directly caused: Google AdSense delivered ads suggesting arrest records to 81-95 percent of black-sounding names compared to much lower rates for white-sounding names.

Indirectly caused: N/A

Inferred additional harm: Widespread differential treatment likely affected millions of black internet users in the United States whose names are racially identifiable.

Civil rights

Reported: The report explicitly discusses potential violations of civil rights, specifically Title VII of the Civil Rights Act of 1964 regarding employment discrimination.

Directly caused: The discriminatory ad delivery resulted in unequal representation based on racial origins, violating principles of equal treatment.

Indirectly caused: N/A

Inferred additional harm: Systemic civil rights impacts on employment equity for black applicants across the United States are highly likely given the ubiquity of search engines in hiring processes.

Psychological

Reported: The report explicitly describes psychological harm, noting that being falsely associated with a criminal record was embarrassing and demeaning.

Directly caused: Latanya Sweeney experienced distress and embarrassment upon seeing ads falsely suggesting she had been arrested.

Indirectly caused: N/A

Inferred additional harm: It is likely that other black individuals whose names triggered these arrest-related ads experienced similar feelings of demeaning treatment, anxiety, and reputational distress, potentially affecting thousands of people.

Epistemic

Reported: The report explicitly describes epistemic harm through the generation of misleading ads suggesting individuals had arrest records when they did not.

Directly caused: AdSense generated false suggestions of arrest histories for individuals like Latanya Sweeney who had clean records.

Indirectly caused: N/A

Inferred additional harm: The widespread generation of false arrest suggestions likely misled numerous employers, colleagues, and acquaintances, eroding the accuracy of online personal information.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 2184

Potential causes

Management

• Prioritization of Revenue: Ad systems maximize revenue and clicks over social fairness.
• Denial of Responsibility: Management shifts blame to advertisers and user behavior.
• Inadequate Risk Assessment: Google failed to assess racial bias risks in ad delivery systems.

Technology

• Smart Optimization Algorithms: Algorithmic optimization prioritizes CTR, reinforcing biased click patterns.
• Black Box Ad Delivery: Complex, opaque ad-serving systems make bias detection difficult.
• Cloud-Caching Strategies: Caching strategies might bias ad delivery toward previously loaded ads.

Data Inputs

• Racially Associated Names: First names predict race, serving as proxy inputs for ad targeting.
• User Click-Through Data: Historical click data contains and perpetuates societal racial biases.
• Ad Template Variations: Advertisers provided templates that interacted with names to trigger ads.

Human Factors

• User Click Biases: Users click arrest ads more frequently for black-sounding names.
• Societal Prejudices: Deeply ingrained racial stereotypes influence user behavior online.
• Lack of Diversity in AI: Homogenous thinking among creators leads to biased technology design.

Process and Methods

• Lack of Algorithmic Auditing: No proactive methods to review algorithms for discriminatory outcomes.
• Flawed Ad Customization Tools: Transparency tools fail to show all inferred interests used for targeting.
• No Proactive Value Design: Engineering processes fail to design for social and legal consequences.

Regulatory Environment

• Lack of Online Ad Regulations: Existing civil rights laws do not easily apply to online ad delivery.
• Opaque Pricing Enforcement: Challenging to enforce anti-discrimination laws in opaque commerce systems.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide detailed, consistent information about Latanya Sweeney's peer-reviewed study, including specific percentages, methodologies, and responses from Google and Instant Checkmate. The core facts of the algorithmic bias are well-documented.
• Ambiguities identified: The exact technical mechanism within Google AdSense (whether caused by advertiser templates, Google's algorithm, or societal click bias) remains unproven due to the proprietary nature of the algorithm.
• Alternative interpretations: The bias could be interpreted as a reflection of societal prejudice (user clicks) rather than a direct failure of Google's technology design.

A landmark 2013 study revealed systemic racial bias in Google's AdSense algorithm, which served arrest-related ads disproportionately for Black-identifying names. While representing a significant civil rights concern and establishing a major precedent for algorithmic bias, the national security impact remains minor and manageable within standard legal, regulatory, and corporate policy frameworks.

• Overall national security impact: Minor
• Response level: Moderate
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing, systemic algorithmic bias concern rather than an active, immediate national security crisis.
• Autonomy: Full autonomy. The ad optimization and delivery systems operated independently to serve ads based on user behavior without real-time human intervention.
• Novelty: First-of-its-kind. This was one of the first major academic studies to document and prove systemic algorithmic racial bias in a major consumer tech platform.

Impact by dimension

• Physical security: Negligible. No impact on physical systems, critical infrastructure, or human safety was reported or indicated.
• Information security: Negligible. The incident involved commercial ad delivery algorithms rather than state-sponsored information warfare or compromise of intelligence capabilities.
• Sovereignty: Negligible. No threat to state authority, territorial control, or core government decision-making processes was identified.
• Economic security: Negligible. While potentially affecting individual employment opportunities, the incident did not threaten strategic industries or national economic stability.
• Societal stability: Minor. The incident revealed systemic racial discrimination in ad delivery, affecting civil rights and societal equity, but is manageable within standard legal and regulatory frameworks.