UK Facial Recognition System Reportedly Exhibits Higher False Positive Rates for Black and Asian Subjects

UK government testing of police facial recognition technology reportedly found significantly higher false positive identification rates for Black and Asian individuals compared with white subjects, with particularly elevated error rates for Black women. The findings reportedly emerged from analysis of retrospective searches of the police national database and were disclosed by the Home Office amid plans for expanded national deployment.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.3 Unequal performance across groups

The facial recognition system exhibited significantly different false positive rates across demographic groups, performing worst for Black and Asian individuals, particularly Black women.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Police forces lobbied to maintain a lower confidence threshold despite knowing it introduced significant demographic bias, prioritizing operational convenience over safe and equitable use.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The demographic bias and high false positive rates were caused by the performance limitations of the deployed AI system, which was an unintended outcome of its operational use.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The report describes the use of facial recognition technology in law enforcement, which is explicitly listed as a high-risk application. The reports state: 'UK forces use the police national database (PND) to conduct retrospective facial recognition searches' and 'Live facial recognition... is currently used by 13 police forces'.

AI system and alleged parties

• AI system: police national database facial recognition technology
• AI purpose: Face Recognition; Identification
• Behaviour type: Assistant
• Alleged developer: Unknown facial recognition technology developers
• Alleged deployer: Metropolitan Police, Law enforcement, Home Office, Government of the United Kingdom, British law enforcement
• Alleged harmed parties: National security and intelligence stakeholders, Minorities in the United Kingdom, General public of the United Kingdom, General public, Epistemic integrity, Black people in the United Kingdom, Asian people in the United Kingdom

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Severe. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Substantial, indirect Substantial
• Civil rights: direct Substantial, indirect Substantial
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Severe, indirect Severe
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Differential treatment

Reported: The reports explicitly describe significant differential treatment of individuals based on race, gender, and age.

Directly caused: The facial recognition software had a false positive rate of 0.04% for white people, compared to 4% for Asian people and 5.5% for Black people (rising to 9.9% for Black women).

Indirectly caused: Black and Asian individuals, particularly Black women, were disproportionately subjected to incorrect matches and potential police intervention due to the biased algorithm.

Inferred additional harm: Ongoing systemic bias in police investigations, leading to disproportionate scrutiny and erosion of trust in law enforcement among minority communities.

Civil rights

Reported: The reports explicitly describe concerns regarding violations of civil rights and lack of safeguards.

Directly caused: Thousands of innocent individuals, disproportionately from ethnic minorities, were wrongly flagged as criminal suspects, infringing on their right to non-discrimination and freedom of movement.

Indirectly caused: Police forces actively lobbied to revert to a biased setting to maintain operational convenience, prioritizing efficiency over the civil rights of protected groups.

Inferred additional harm: Widespread, undocumented infringements on civil liberties and privacy for individuals whose images were processed by the biased system without their knowledge.

Privacy

Reported: The reports describe privacy concerns related to scanning faces and cross-referencing them against databases.

Directly caused: Retrospective searches match images from CCTV or mobile phones against watchlists on the national database containing over 19 million custody photos.

Indirectly caused: Plans to expand the system to access other databases, including passport and driving licence images, without established robust safeguards.

Inferred additional harm: Mass surveillance and unauthorized processing of biometric data of millions of citizens, violating their expectation of privacy.

People affected

• Occurrences reported: 1
• People reportedly harmed: 2000
• People reportedly exposed: 19000000

Potential causes

Management

• Prioritizing Leads over Fairness: Police leadership prioritized operational volume over reducing racial bias.
• Poor Communication with Communities: Management failed to share known bias issues with affected communities.
• Inadequate Risk Mitigation: Failed to establish robust safeguards before wide system rollout.

Technology

• Biased Matching Algorithm: Algorithm shows significantly higher false positive rates for minorities.
• Low Algorithm Confidence Settings: Lowering confidence thresholds to get more leads increased false positives.
• Performance Decline on Women: The software performs worse on women and younger demographics.

Data Inputs

• Custody Photo Database Bias: Matching against 19 million custody images introduces demographic skew.
• Poor Quality Probe Images: CCTV and mobile photos used as inputs can degrade matching accuracy.
• Unbalanced Training Datasets: The underlying AI model was trained on demographically unrepresentative data.

Human Factors

• Preference for Quantity over Quality: Police preferred more investigative leads despite higher false positive rates.
• Risk of Cognitive Bias in Review: Officers reviewing matches may trust the AI suggestion too readily.
• Inadequate User Training: Officers lacked sufficient training to mitigate algorithmic bias effects.

Process and Methods

• Reverting to Low Thresholds: Reversing a high threshold decision to restore operational search volume.
• Inadequate Pre-Deployment Testing: Technology was deployed into operational policing without safeguards.
• Delayed Public Disclosure: Known system bias was kept quiet for over a year before publication.

Regulatory Environment

• Absence of Strict National Standards: No robust national standards exist to regulate facial recognition use.
• Weak Independent Scrutiny: Lack of independent oversight bodies to audit police AI systems.
• Lack of Clear Legal Framework: Deployment proceeded without detailed legislative debate or restrictions.

Information quality

• Classification confidence: High
• Reason for confidence: The reports are highly detailed, citing official research from the National Physical Laboratory (NPL) with specific statistical figures on false positive rates across different demographic groups. The accounts from the Home Office, police representatives, and civil rights groups are consistent regarding the existence of the bias and the operational decisions made around the confidence thresholds.
• Ambiguities identified: The exact number of historical false positives that led to actual police interventions is not fully quantified, though the Met Police provides a small sample (10 false alerts, 6 stops).
• Alternative interpretations: None. The bias and the decision to revert the threshold are clearly documented.

UK police forces deployed facial recognition technology with significant demographic bias, particularly affecting Black and Asian individuals. Despite internal findings, forces lobbied to maintain a lower confidence threshold for operational convenience, leading to civil rights concerns and potential erosion of public trust in domestic law enforcement.

• Overall national security impact: Substantial
• Scope: Single nation
• Primary target: United Kingdom
• Alleged perpetrator: UK police forces

Threat characteristics

• Imminence: Long-term. Represents an ongoing, systemic policy and technology deployment concern rather than an immediate tactical threat.
• Autonomy: Human-supervised. The AI system generates potential matches, but human officers are required to verify the results before taking law enforcement action.
• Novelty: Established threat. Demographic and racial bias in facial recognition algorithms is a well-documented and established issue in AI deployment.

Impact by dimension

• Physical security: Negligible. No physical security threats, kinetic attacks, or critical infrastructure compromise were reported in relation to this domestic law enforcement deployment.
• Information security: Negligible. No active foreign intelligence compromise, systematic disinformation, or information warfare operations were identified in this incident.
• Sovereignty: Minor. Domestic policing operations and decision-making processes were affected by algorithmic bias, but without threatening state authority or core constitutional processes.
• Economic security: Negligible. No strategic technology theft, financial system manipulation, or critical supply chain disruptions were reported.
• Societal stability: Substantial. Systematic demographic bias in facial recognition disproportionately targeted ethnic minorities, undermining civil liberties and potentially eroding public trust in policing.