Common Biases of Vector Embeddings

Researchers from Boston University and Microsoft Research, New England demonstrated gender bias in the most common techniques used to embed words for natural language processing (NLP).

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.1 Unfair discrimination and misrepresentation

The word embeddings exhibit and amplify gender stereotypes, which represents unfair representation and stereotyping of demographic groups.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The gender bias and its amplification are caused by the mathematical representations within the AI system's word embeddings, which was an unintended outcome of training on Google News data.

EU AI Act risk tier

• Risk tier: 4 Minimal or No Risk

Minimal or No Risk: The report describes a word embedding model, which is a general-purpose machine learning tool with no inherent high-risk application specified in the text.

AI system and alleged parties

• AI system: word embeddings trained on Google News articles (Google)
• AI purpose: Data Grouping; Translation
• Behaviour type: Tool
• Alleged developer: Microsoft Research, Google, Boston University
• Alleged deployer: Microsoft Research, Boston University
• Alleged harmed parties: Women, Minority Groups

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Substantial
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Differential treatment

Reported: The report explicitly describes how word embeddings trained on Google News articles exhibit and can amplify female/male gender stereotypes.

Directly caused: The embeddings directly associate gender-neutral words with specific genders, such as associating 'receptionist' with female, reflecting gender stereotypes.

Indirectly caused: The widespread use of these biased embeddings in downstream machine learning applications tends to amplify these gender biases.

Inferred additional harm: It is likely that systems using these embeddings (e.g., resume screening, search engines) indirectly perpetuate gender discrimination against women in real-world hiring or search contexts, though specific numbers are not provided.

People affected

• Occurrences reported: 1

Potential causes

Technology

• Biased Word Embedding Vectors: Word embeddings capture gender stereotypes geometrically in vector space.
• Bias Amplification in ML: Blind application of ML models amplifies biases present in embeddings.

Data Inputs

• Biased Training Data: Google News articles contain inherent female and male gender stereotypes.

Human Factors

• Blind Application of ML: Users deploy models without considering or checking for underlying biases.

Process and Methods

• Lack of Debiasing Workflows: Embeddings are used without applying algorithms to remove stereotypes.
• Missing Bias Metrics: Lack of standard metrics to evaluate direct and indirect embedding bias.

Information quality

• Classification confidence: High
• Reason for confidence: The report is a clear, peer-reviewed scientific paper detailing the specific technical issue of gender bias in Google News word embeddings. The methodology, findings, and implications are explicitly stated, leaving little ambiguity about the nature of the AI system and the bias identified.

An academic study identified systematic gender bias in Google News word embeddings. While the research highlights important ethical and technical concerns regarding the propagation of stereotypes in downstream AI applications, the incident is entirely academic and poses negligible direct threat to national security.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: No clear primary
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The issue represents an ongoing, systemic technological challenge regarding AI bias rather than an active, imminent national security crisis.
• Autonomy: Human-controlled. Word embeddings are static mathematical representations used as tools by human developers within larger software systems.
• Novelty: Evolved capability. While data bias is an established issue, mathematically demonstrating and quantifying gender bias geometrically within word embeddings represented an evolution in understanding.

Impact by dimension

• Physical security: Negligible. The incident is an academic study on algorithmic bias in word embeddings and poses no threat to physical systems, critical infrastructure, or human safety.
• Information security: Negligible. There is no evidence of intelligence compromise, classified data theft, or active information warfare operations associated with this research.
• Sovereignty: Negligible. The discovery of gender bias in language models does not impact state authority, electoral systems, or core government operations.
• Economic security: Negligible. While the bias affects commercial NLP models, it does not represent a strategic economic threat, technological theft, or critical supply chain vulnerability.
• Societal stability: Negligible. Although the study highlights that word embeddings can perpetuate gender stereotypes in downstream applications, the immediate national security impact on societal stability is negligible.