London Metropolitan Police's Facial Recognition Technology Reportedly Misidentified Shaun Thompson as Suspect Leading to Arrest

Sometime in February 2024, Shaun Thompson is reported to have walked by one of the London Metropolitan Police's facial recognition technology vans near London Bridge. He was almost immediately arrested because the technology is reported to have misidentified him as a suspect in an unrelated and unspecified crime.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The incident was primarily caused by the facial recognition systems failing to perform reliably, resulting in false positive matches and mistaken identity.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Police officers and retail staff immediately acted on the AI's alerts to detain and accuse individuals without sufficient independent verification.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The false positive matches were caused by the AI system's algorithmic errors after it was deployed in public and retail spaces.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The AI system is used for law enforcement applications, specifically live facial recognition in public spaces, which is explicitly listed as a High Risk application under the EU AI Act.

AI system and alleged parties

• AI system: Facewatch
• AI purpose: Face Recognition; Identification
• Behaviour type: Tool
• Alleged developer: Metropolitan Police Service
• Alleged deployer: Metropolitan Police Service
• Alleged harmed parties: Shaun Thompson, General public

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Minor, indirect Substantial
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Differential treatment

Reported: The report explicitly describes differential treatment of individuals caused directly by the incident.

Directly caused: The AI system flagged specific individuals, leading to them being singled out, stopped, searched, and detained while others passed freely.

Civil rights

Reported: The report explicitly describes violations of civil rights caused directly by the incident.

Directly caused: Mr. Thompson was detained for 20-30 minutes and asked for fingerprints and passport, violating his freedom of movement and personal liberty. The shopper had her bag searched and was banned from stores.

Inferred additional harm: The widespread deployment of live facial recognition scans thousands of citizens without consent, potentially infringing on collective privacy and civil liberties.

Privacy

Reported: The report explicitly describes privacy violations caused directly and indirectly by the incident.

Directly caused: Mr. Thompson's biometric data was scanned and he was asked to provide fingerprints. The shopper's bag was searched.

Indirectly caused: The system scans and processes the biometric facial data of thousands of passersby without their explicit consent.

Inferred additional harm: Mass scanning of public spaces represents a large-scale intrusion into the privacy of thousands of citizens.

Psychological

Reported: The report explicitly describes psychological distress caused directly by the false positive matches.

Directly caused: The anonymous shopper was left 'crying and crying the entire journey home' and feared her life would never be the same due to being looked at as a shoplifter. Mr. Thompson felt the experience was 'intrusive' and felt treated as 'guilty until proven innocent'.

Inferred additional harm: Other individuals falsely flagged by these systems likely experienced similar anxiety, public humiliation, and distress.

People affected

• Occurrences reported: 2
• People reportedly harmed: 2
• People reportedly exposed: 2

Potential causes

Management

• Rapid Technology Scaling: Met Police scaled deployment from 9 to 67 times without fixing error rates.
• Prioritizing Speed Over Accuracy: Focusing on rapid arrests led to overlooking high false positive risks.

Technology

• High False Positive Rate: One in 40 alerts was a false positive, causing wrongful detentions.
• Family Resemblance Mismatch: System struggled to differentiate individuals with strong family resemblance.
• Flawed Biometric Matching: Facewatch algorithm incorrectly matched innocent shopper to shoplifter.

Data Inputs

• Poor Watchlist Image Quality: Low-quality database photos can lead to algorithmic mismatching.
• Suboptimal Camera Angles: Roof-mounted van cameras captured images prone to distortion.

Human Factors

• Automation Bias: Operators trusted algorithmic alerts without sufficient skepticism.
• Immediate Accusation by Staff: Store worker accused shopper of theft based solely on system alert.

Process and Methods

• Lack of Pre-Confrontation Checks: No rigorous human verification step before detaining flagged targets.
• Intrusive Identity Verification: Detainees forced to provide fingerprints or passports to prove innocence.

Regulatory Environment

• Absence of Clear Legislation: Lack of specific laws governing facial recognition creates legal uncertainty.
• Insufficient Oversight: No independent watchdog regulating private use of biometric databases.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, firsthand accounts of the individuals affected by the facial recognition errors, along with official statistics from the Metropolitan Police regarding error rates (1 in 40 alerts being false positives). The roles of the AI systems (Facewatch and Met Police LFR) are explicitly stated, and the resulting harms are well-documented.
• Ambiguities identified: The exact technical reasons for the false matches are not fully detailed, beyond a brief mention of a potential 'family resemblance' in Mr. Thompson's case.

The deployment of live facial recognition by UK police and private retailers has led to incidents of mistaken identity, resulting in wrongful detentions and privacy violations. While raising significant domestic civil liberties and human rights concerns regarding mass surveillance, the national security impact remains minor as it does not involve foreign adversaries, kinetic threats, or critical infrastructure compromise.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United Kingdom
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The deployment of live facial recognition represents an ongoing regulatory and strategic policy concern rather than an immediate national security crisis.
• Autonomy: Human-supervised. The AI system operates autonomously to scan and match faces, but human officers and retail staff make the final decision to detain or intervene.
• Novelty: Established threat. Mistaken identity and false positives in biometric facial recognition systems are well-documented, established technical limitations.

Impact by dimension

• Physical security: Negligible. No physical infrastructure damage, kinetic attacks, or threats to physical safety were reported in connection with the facial recognition errors.
• Information security: Minor. Deployment of live facial recognition represents a mass surveillance capability, but there is no evidence of foreign intelligence compromise or active information warfare.
• Sovereignty: Minor. Domestic law enforcement operations are affected by false positive alerts, but there is no threat to state authority, electoral processes, or territorial control.
• Economic security: Negligible. No threats to strategic industries, financial systems, or national technological competitive advantage are indicated by these retail and local policing incidents.
• Societal stability: Minor. The incidents demonstrate minor but notable impacts on civil liberties and privacy, including wrongful detentions and public distress, raising public concern over mass surveillance.