New York Detective Misused Woody Harrelson's Face to Perform Face Recognition Search

When the facial recognition search for a CVS theft suspect's face returned no useful matches due to the surveillance footage being obscured and highly pixelated, a New York City police detective continued the face search using Woody Harrelson's face allegedly due to his resemblance to the suspect's face, eventually leading to the arrest of an unknown victim.

Risk classification

• Primary risk domain: 5 Human-Computer Interaction
• Primary risk subdomain: 5.1 Overreliance and unsafe use

Law enforcement officers are unsafely deploying facial recognition systems in critical situations by inputting highly flawed data (sketches, celebrity photos) and overrelying on the results to make arrests without proper independent corroboration.

Additional risk subdomains

• 7.3 Lack of capability or robustness: The facial recognition systems fail to perform reliably when subjected to non-photographic inputs like forensic sketches, yielding correct matches only 4% to 7% of the time.

Causal factors

• Entity: Human
• Intent: Unintentional
• Timing: Post-deployment

The risk of wrongful arrest occurs due to an unexpected outcome (misidentification) from pursuing the goal of identifying suspects using flawed inputs.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The report describes facial recognition systems used in law enforcement applications to identify criminal suspects, which is explicitly classified as High Risk under the EU AI Act.

AI system and alleged parties

• AI system: NYPD facial recognition system
• AI purpose: Face Recognition; Identification
• Behaviour type: Tool
• Alleged developer: unknown
• Alleged deployer: New York Police Department, Facial Identification Section
• Alleged harmed parties: unknown

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Severe. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Minor, indirect Minor
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Severe, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Differential treatment

Reported: Yes, the report describes how detectives pasted features of a 'Black Male Model' over a suspect's mouth, indicating potential racial dimensions in how probe images are manipulated.

Directly caused: NYPD detectives edited a probe image by searching for 'Black Male Model' and pasting his lips over the suspect's mouth.

Inferred additional harm: The practice of manipulating photos and using biased databases likely leads to systemic differential treatment and higher rates of misidentification for Black suspects.

Civil rights

Reported: Yes, the report explicitly discusses the deprivation of liberty, wrongful arrests, and the failure to protect defendants' due process rights.

Directly caused: Individuals were arrested and deprived of their liberty based on unreliable facial recognition matches (e.g., the Woody Harrelson doppelganger case).

Indirectly caused: Defendants are denied due process because defense attorneys are not provided with information about the use of 'virtual probes' or facial recognition in their clients' cases.

Inferred additional harm: Thousands of individuals arrested under these systems may have had their civil rights violated through unlawful detentions and lack of due process.

Privacy

Reported: Yes, the report mentions police searching photos against driver's license and police databases without clear rules.

Directly caused: Police departments run millions of photos of citizens (including driver's licenses) through facial recognition databases without their consent.

Inferred additional harm: Mass surveillance and unauthorized biometric searches violate the privacy expectations of millions of citizens whose photos are in these databases.

Epistemic

Reported: Yes, the creation of 'virtual probes' and edited photos (pasting eyes, mirroring faces, using sketches) represents the fabrication of fake biometric evidence.

Directly caused: NYPD and other agencies graphically altered photos, fabricating facial identity points (e.g., pasting open eyes or lips from Google searches) to submit to the AI.

Indirectly caused: The AI system returned high confidence matches (e.g., 95%) based on fabricated data, misleading investigators into believing they had a reliable match.

Inferred additional harm: This practice corrupts the integrity of forensic evidence used in the criminal justice system, leading to a broader erosion of truth in legal proceedings.

People affected

• Occurrences reported: 1
• People reportedly harmed: 6
• People reportedly exposed: 2878

Potential causes

Management

• Prioritizing Leads Over Accuracy: Management allowed questionable inputs to generate fast investigative leads.
• Inadequate Officer Training: Lack of training on limits of facial recognition and data manipulation.
• Lack of System Accountability: Departments failed to track search metrics, accuracy, and resulting arrests.

Technology

• FR Algorithms Misuse: Algorithms are not designed to match forensic sketches or edited photos.
• Lack of Input Validation: The system does not reject low-quality or heavily modified probe photos.
• High Confidence for Edited Photos: Algorithms return high confidence scores on heavily modified/fabricated faces.

Data Inputs

• Use of Celebrity Doppelgangers: Detectives submitted celebrity photos instead of actual suspect images.
• Submission of Forensic Sketches: Forensic sketches were used as probe images despite extremely low accuracy.
• Fabricated Facial Features: Photos were edited to add features from unrelated online search results.

Human Factors

• Overreliance on FR System: Detectives assumed the FR system's output was highly reliable.
• Cognitive Bias in Matching: Analysts confirmed matches based on subjective and flawed inputs.
• Creative Probe Selection: Detectives used celebrity lookalikes when suspect photos returned no matches.

Process and Methods

• Flawed Image Editing Practices: Detectives edited photos with tools like Clone Stamp, fabricating features.
• Lack of Corroboration Standards: No clear guidance on what independent evidence is needed to back up a match.
• Inadequate Human Review: Subsequent reviews were conducted against edited photos rather than originals.

Regulatory Environment

• No Rules on Probe Photos: No legal rules exist governing what images police can submit to FR systems.
• Lack of Defense Disclosure: Defense attorneys are routinely denied access to FR search details.
• Absence of Federal Standards: Lack of federal standards or oversight on police facial recognition usage.

Information quality

• Classification confidence: High
• Reason for confidence: The report is highly detailed, citing specific cases (like the Woody Harrelson incident), police records, scientific studies, and official recommendations. The roles of the AI systems and the human operators are clearly delineated.
• Ambiguities identified: The exact algorithms used in each specific arrest (e.g., whether AWS Rekognition or Cognitec was used in the Woody Harrelson case specifically) are not individually mapped to every single anecdote, but the overall practice is well-documented.
• Alternative interpretations: One could argue the failure is entirely human (police misconduct) rather than an AI safety issue, but the interaction between human editing and algorithmic confidence scores makes it a socio-technical AI safety incident.

The widespread and unregulated use of flawed 'probe images' by law enforcement agencies in facial recognition systems has led to wrongful arrests and systemic civil rights violations. While it poses no kinetic or foreign intelligence threat, it represents a substantial challenge to societal stability, privacy, and due process within the domestic justice system.

• Overall national security impact: Substantial
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: U.S. Law Enforcement Agencies

Threat characteristics

• Imminence: Long-term. Represents an ongoing systemic practice and regulatory challenge rather than an immediate, acute national security crisis.
• Autonomy: Human-controlled. The AI acts as an investigative tool, with human operators manually editing inputs and making the final decisions regarding arrests.
• Novelty: Evolved capability. While facial recognition misuse is known, the active fabrication of biometric inputs (virtual probes) represents an evolved and highly concerning practice.

Impact by dimension

• Physical security: Negligible. No physical security threats, kinetic attacks, or critical infrastructure impacts are associated with this incident.
• Information security: Negligible. The incident does not involve foreign intelligence compromise, espionage, or state-sponsored information operations.
• Sovereignty: Minor. Minor impact on local judicial and law enforcement integrity due to unregulated practices, but does not threaten state authority or border control.
• Economic security: Negligible. No significant threats to economic stability, strategic technology theft, or loss of technological competitive advantage.
• Societal stability: Substantial. The systemic use of flawed inputs in facial recognition leads to wrongful arrests, erosion of privacy, and significant civil rights and due process violations.