Facebook's Auto-Generated Targeting Ad Categories Contained Anti-Semitic Options

Facebook's automated advertising categories generated using users' declared interests contained anti-Semitic categories such as "Jew hater" and "How to burn Jews" which were listed as fields of study.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.2 Exposure to toxic content

The AI systems generated and exposed toxic, anti-Semitic hate-speech categories within the advertiser portals and allowed ads to be targeted at these groups.

Additional risk subdomains

• 1.1 Unfair discrimination and misrepresentation: The automated systems enabled potential discrimination by allowing advertisers to target or exclude users based on protected characteristics like religion.
• 7.3 Lack of capability or robustness: The algorithms lacked the capability or robust filtering mechanisms to distinguish benign academic topics from explicit hate speech.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The risk was caused by the automated ad-targeting algorithms of Facebook and Google generating hate-speech categories, which was an unexpected and unintentional outcome of their post-deployment operation.

EU AI Act risk tier

• Risk tier: 4 Minimal or No Risk

Minimal or No Risk: The system is an automated ad targeting and keyword suggestion algorithm, which generally falls under minimal risk as it does not belong to the prohibited or high-risk categories defined in the Act.

AI system and alleged parties

• AI system: Facebook ad-targeting algorithm, Google keyword suggestions tool (Google, Meta)
• AI purpose: Ad Delivery; Data Grouping
• Behaviour type: Assistant
• Alleged developer: Facebook
• Alleged deployer: Facebook
• Alleged harmed parties: Jewish people

Harm severity

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Minor
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Minor, indirect Minor
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes

Directly caused: Facebook's algorithm automatically generated hate-speech categories like 'Jew hater' and 'How to burn Jews' in its advertiser portal, and Google's keyword tool suggested similar anti-Semitic phrases.

Indirectly caused: ProPublica successfully ran three promoted posts targeting these categories, reaching 5,897 people.

Inferred additional harm: Other malicious advertisers may have utilized similar automatically generated offensive categories to run toxic ads before the fields were disabled.

Differential treatment

Reported: Yes

Directly caused: The system allowed advertisers to target or exclude users based on sensitive, protected characteristics, specifically religion (anti-Semitism).

Indirectly caused: N/A

Inferred additional harm: The existence of these categories enabled systematic targeting and potential harassment of Jewish users or promotion of anti-Semitic content to sympathetic audiences.

Civil rights

Reported: Yes

Directly caused: The ad targeting system violated Facebook's own community standards against hate speech and discrimination based on protected characteristics.

Indirectly caused: ProPublica notes that similar ad targeting practices in housing had previously allowed discrimination against racial minorities, violating the Fair Housing Act.

Inferred additional harm: The facilitation of anti-Semitic targeting represents a systemic threat to the civil rights of Jewish individuals to be free from discrimination.

People affected

• Occurrences reported: 1
• People reportedly exposed: 8171

Potential causes

Management

• Prioritizing Growth Over Safety: Management focused on scaling the ad platform without safety guardrails.
• Inadequate Risk Assessment: Failed to anticipate risk of hate speech in automated targeting fields.

Technology

• Algorithmic Category Generation: Algorithms converted user profile text into targetable ad categories.
• Automated Recommendation Systems: System suggested related offensive terms to expand target audience size.
• Lack of Automated Hate Speech Filters: No automated filters blocked offensive terms from becoming ad categories.

Data Inputs

• User-Generated Profile Data: Users entered offensive terms in fields like education or employer.
• Implicit User Activity Tracking: System derived interests from implicit user behavior without validation.

Human Factors

• Lack of Human Oversight: No human review of automatically generated ad categories before deployment.
• Advertiser Exploitation: Advertisers deliberately targeted hate-based groups for campaigns.

Process and Methods

• Flawed Ad Review Process: Self-service system approved offensive ads automatically within 15 minutes.
• Hands-off Ad Model: Business model prioritized automated scaling over manual content verification.

Regulatory Environment

• Lack of External Oversight: No regulatory standards governed automated ad-targeting categories.

Information quality

• Classification confidence: High
• Reason for confidence: The reports from ProPublica, The Guardian, and TechCrunch are highly detailed, consistent, and include direct confirmation and statements from Facebook and Google executives. The mechanics of how the algorithm generated the categories from user profile inputs are clearly explained.
• Ambiguities identified: None of significance; the technical cause (algorithmic aggregation of self-reported profile fields) is clearly identified.
• Alternative interpretations: None.

An investigation revealed that Facebook and Google automated ad systems generated anti-Semitic targeting categories from user data without human oversight. While presenting potential vulnerabilities for information operations and discrimination, the incident was quickly mitigated by the platforms and did not escalate into a direct national security crisis.

• Overall national security impact: Minor
• Response level: Moderate
• Scope: Multiple nations
• Primary target: No clear primary
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing systemic vulnerability in automated content moderation rather than an active, immediate crisis.
• Autonomy: Full autonomy. The algorithms automatically processed user data and generated targetable categories without any human oversight or approval.
• Novelty: Evolved capability. Demonstrated a significant advancement in how automated systems can unintentionally generate and scale harmful targeting categories.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or human safety threats were involved in this digital ad-targeting incident.
• Information security: Minor. The platforms facilitated automated targeting of divisive and offensive categories, presenting a vulnerability for potential information operations, though no active foreign state campaign was identified.
• Sovereignty: Negligible. No direct impact on state authority, government decision-making, or core constitutional processes was reported.
• Economic security: Negligible. No significant threat to national economic stability, strategic industries, or technological competitive advantage.
• Societal stability: Minor. The automated creation of anti-Semitic targeting categories facilitated potential discrimination and toxic content exposure, but was quickly mitigated and did not cause widespread unrest.