Amazon's Rekognition Falsely Matched Members of Congress to Mugshots

Rekognition's face comparison feature was shown by the ACLU to have misidentified members of congress, and particularly members of colors, as other people who have been arrested using a mugshot database built on publicly available arrest photos.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.3 Unequal performance across groups

The facial recognition system demonstrated unequal accuracy across demographic groups, with a disproportionately higher rate of false matches for people of color.

Additional risk subdomains

• 7.3 Lack of capability or robustness: The system failed to perform reliably by generating 28 false matches against a mugshot database under default settings.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The false matches were generated by the deployed Amazon Rekognition AI system, representing an unintentional failure of the technology's accuracy.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The report describes a facial recognition system, which is explicitly classified under 'Law enforcement applications, including facial recognition' as a High Risk AI system under the EU AI Act.

AI system and alleged parties

• AI system: Rekognition (Amazon)
• AI purpose: Face Recognition; Identification
• Behaviour type: Tool
• Alleged developer: Amazon
• Alleged deployer: Amazon
• Alleged harmed parties: Rekognition users, arrested people

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Negligible, indirect Minor
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Differential treatment

Reported: Yes, the report explicitly describes differential treatment in the system's performance across demographic groups.

Directly caused: The facial recognition software disproportionately misidentified people of color, with nearly 40 percent of the false matches being people of color, despite them making up only 20 percent of Congress.

Civil rights

Reported: Yes, the report explicitly discusses the threat to civil rights and liberties.

Directly caused: The false matches of 28 members of Congress to arrest photos falsely associated them with criminal records, impacting their reputational standing.

Indirectly caused: The report notes that face surveillance technology threatens to chill First Amendment-protected activities like protest and religious practice, and subjects vulnerable groups to potential government abuse.

People affected

• Occurrences reported: 1
• People reportedly harmed: 28
• People reportedly exposed: 535

Potential causes

Management

• Aggressive Marketing: Amazon markets face surveillance to police before addressing accuracy issues.
• Neglecting Ethical Feedback: Shareholder and civil rights group warnings about harms were ignored.

Technology

• Inherent Algorithmic Bias: Academic research shows lower accuracy for darker-skinned faces and women.
• Low Default Confidence Threshold: Using default match settings allowed false positives to be generated easily.

Data Inputs

• Biased Mugshot Database: The 25,000 arrest photos database contains historical demographic imbalances.
• Unstandardized Input Photos: Public photos of Congress members may differ in lighting and quality.

Human Factors

• Automation Bias: Users may blindly trust facial recognition matches without verification.

Process and Methods

• Lack of Rigorous Testing: Insufficient pre-deployment testing on diverse demographic groups.
• Flawed Default Settings: Default settings are not optimized for high-stakes law enforcement use.

Regulatory Environment

• Absence of Federal Regulations: No federal laws govern or restrict the deployment of face surveillance.
• Lack of Oversight: Police departments adopt the technology without public debate or oversight.

Information quality

• Classification confidence: High
• Reason for confidence: The report from the ACLU is highly detailed regarding the methodology of the test, the specific AI system used (Amazon Rekognition), the exact number of false matches (28), and the demographic breakdown of the errors. There is little ambiguity about the events of the test itself.

The ACLU test of Amazon Rekognition exposed significant demographic bias and false-positive rates in commercial facial recognition software by falsely matching US Congress members to mugshots. While the test itself did not disrupt national security, it highlighted substantial civil rights risks and potential for biased law enforcement actions, fueling debate over domestic surveillance policy.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: ACLU

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding AI bias and civil liberties rather than an active crisis.
• Autonomy: Human-controlled. The AI operates as a tool performing facial matching based on direct human input and settings.
• Novelty: Evolved capability. Significantly advanced public awareness and documented evidence of demographic bias in commercial facial recognition systems.

Impact by dimension

• Physical security: Negligible. No impact on physical systems, critical infrastructure, or human safety was reported in this incident.
• Information security: Negligible. No evidence of information warfare, intelligence compromise, or systematic manipulation of information by hostile actors.
• Sovereignty: Negligible. No disruption to core government operations or sovereignty; the test was conducted publicly by a civil liberties group.
• Economic security: Negligible. No significant threat to economic stability, strategic industries, or technological competitive advantage.
• Societal stability: Minor. Demonstrated potential for algorithmic bias and unequal performance against people of color, raising civil rights and privacy concerns, but limited to a controlled test.