Fatalities Reportedly Occur Despite VioGén Algorithm's Low or Negligible Risk Scores

The VioGén algorithm was designed to help Spanish police assess and prioritize the risk of repeat domestic violence incidents. However, its low-risk assessment of Lobna Hemid reportedly led to inadequate protection; her husband murdered her. Since 2007, 247 women have been killed after being assessed by VioGén. A review of 98 homicides found that 55 of the slain women were scored as negligible or low risk.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The VioGén system failed to perform reliably under real-world conditions, frequently miscalculating domestic violence risk levels and leading to fatal errors.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Police officers and judges heavily relied on the algorithm's risk scores, accepting them 95% of the time and failing to provide protection to victims.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The fatal outcomes resulted from the VioGén AI system's incorrect low-risk classifications post-deployment, which was an unintended consequence of a system designed to protect victims.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The system is used in law enforcement and safety-critical public services to assess risk of violence, which has 'significant implications for safety, fundamental rights, and public interests'.

AI system and alleged parties

• AI system: VioGén
• AI purpose: Predictive Policing; Judicial Decision Support
• Behaviour type: Assistant
• Alleged developer: VioGén algorithm development team, Spanish law enforcement agencies, Spanish Interior Ministry
• Alleged deployer: Spanish law enforcement agencies, Spanish Interior Ministry
• Alleged harmed parties: Women in Spain, Stefany González Escarraman, Spanish general public, María, Luz, Lobna Hemid, Eva Jaular, 247 women in Spain (unnamed)

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Severe
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Minor
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Substantial, indirect Substantial
• Civil rights: direct Negligible, indirect Substantial
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Physical

Reported: The report explicitly describes fatal physical harm to multiple women and one child.

Indirectly caused: At least 247 women have been killed by their partners since 2007 after being assessed by VioGén, including Lobna Hemid, [PERSON_011] Escarraman, and [PERSON_017], whose 11-month-old daughter was also killed.

Inferred additional harm: It is highly likely that thousands of other women suffered non-fatal physical injuries due to being incorrectly classified as low risk and denied protection.

Financial

Reported: The report mentions one legal verdict against the state but does not quantify financial losses.

Indirectly caused: In 2020, the family of [PERSON_011] Escarraman won a financial verdict against the state for failing to protect her, though the amount is not specified.

Inferred additional harm: Families of the deceased likely faced significant funeral costs, legal fees, and loss of income, while the state faced legal liabilities.

Differential treatment

Reported: The report describes differential treatment in resource allocation based on the AI's risk scores, and potential systemic bias against immigrant women.

Directly caused: The VioGén system directly determined which victims received active police protection and which received minimal support based on its risk scores.

Indirectly caused: Family members noted that underlying discrimination in Spain's criminal system may have overlooked violence against immigrant Muslim women like Lobna Hemid.

Inferred additional harm: Systemic unequal protection likely disproportionately affected vulnerable groups who faced barriers in communicating complete information to the police.

Civil rights

Reported: The report describes violations of the fundamental right to life and protection from violence.

Indirectly caused: The state's reliance on flawed AI risk assessments resulted in a failure to protect the lives of at least 247 women, violating their basic human rights.

Inferred additional harm: Thousands of women's civil rights to safety and equal protection under the law were compromised due to incorrect algorithmic classifications.

Psychological

Reported: The report describes severe psychological trauma experienced by the victims' families and children.

Indirectly caused: The children of the victims witnessed severe domestic violence and, in some cases, the fatal stabbings of their mothers, causing immense psychological trauma.

Inferred additional harm: Widespread psychological distress, anxiety, and fear are highly likely among the thousands of domestic violence victims who were denied adequate protection due to low risk scores.

People affected

• Occurrences reported: 1
• People reportedly harmed: 248
• People reportedly exposed: 92000

Potential causes

Management

• Resource Prioritization Bias: Using the algorithm to ration scarce protection resources instead of funding.
• Inadequate Officer Training: Failure to provide sufficient training for handling algorithmic outputs.
• Lack of System Transparency: Management failed to release comprehensive data on system effectiveness.

Technology

• Flawed Risk Assessment Logic: Rigid statistical model fails to capture complex domestic abuse escalation.
• Inability to Predict Rare Events: Algorithm struggles to accurately predict rare high-consequence homicides.
• Opaque Algorithm Logic: Proprietary algorithm prevents outside audits and peer review of its safety.

Data Inputs

• Incomplete Victim Disclosures: Fear, shame, and immigration status led victims to withhold key details.
• Binary Question Restrictions: A 35-question yes/no form fails to capture nuanced threat details.
• Missing Collateral Information: Police missed critical clues by not interviewing neighbors or school staff.

Human Factors

• Automation Bias by Police: Officers accepted the VioGen score 95 percent of the time without questioning.
• Overwhelmed Police Staff: Overburdened agents with high caseloads relied heavily on the algorithm.
• Judicial Reliance on Scores: Judges denied restraining orders based primarily on low VioGen risk scores.

Process and Methods

• Inadequate Interview Methods: Police-led questioning of tired victims without trusted support or experts.
• Lack of Independent Audits: Government refused to allow external audits of the VioGen algorithm.
• Insufficient Follow-up Protocols: Retesting protocols failed to detect escalating threats or broken orders.

Regulatory Environment

• Lack of Algorithmic Oversight: No regulatory framework to audit or govern predictive AI in law enforcement.
• No Mandated Legal Support: Regulations did not guarantee specialized legal or psychological support.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides extensive, detailed statistics from Spain's Interior Ministry and judicial reviews, along with multiple specific case studies of victims, police officers, and creators of the system. The role of the algorithm and the resulting real-world harms are clearly documented.
• Ambiguities identified: The exact mathematical formula of the VioGén algorithm is not public, and the government has refused to make it available for outside audit.
• Alternative interpretations: The failures could be interpreted primarily as human errors in data entry or police resource constraints rather than algorithmic failure, though the report emphasizes the systemic reliance on the algorithm's scores.

The Spanish government's use of the VioGen algorithm for domestic violence risk assessment resulted in systemic failures, leading to incorrect low-risk classifications and subsequent deaths of protected individuals. While representing a tragic failure of state protection and human rights, the incident remains a domestic policing issue with minor national security implications.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: Spain
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing, systemic public policy and policing challenge rather than an immediate national security crisis.
• Autonomy: Human-supervised. The AI generates risk scores to assist police officers, who retain the formal authority to override recommendations, though they rarely do.
• Novelty: Established threat. Algorithmic bias and risk-assessment failures in law enforcement are well-documented, and this specific system has operated since 2007.

Impact by dimension

• Physical security: Negligible. No threats to physical systems, critical infrastructure, kinetic targeting, or autonomous weapons deployment are indicated.
• Information security: Negligible. No evidence of intelligence compromise, disinformation campaigns, or classified data theft.
• Sovereignty: Minor. The algorithm affects police resource allocation and judicial decision-making, representing minor disruption to standard government operations but no threat to state authority.
• Economic security: Negligible. No impact on strategic industries, financial systems, or technological competitive advantage.
• Societal stability: Minor. Systemic failure to protect domestic violence victims led to hundreds of deaths, violating fundamental rights to safety, but does not threaten overall social cohesion or state stability.