US CBP App's Failure to Detect Black Faces Reportedly Blocked Asylum Applications

CBP One's facial recognition feature was reportedly disproportionately failing to detect faces of Black asylum seekers from Haiti and African countries, effectively blocking their asylum applications.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.3 Unequal performance across groups

The facial recognition system performs with significantly lower accuracy for darker-skinned individuals, directly preventing them from completing their asylum applications.

Additional risk subdomains

• 1.1 Unfair discrimination and misrepresentation: The technical failure results in systemic exclusion and unequal treatment of Black asylum seekers at the border based on race.
• 7.3 Lack of capability or robustness: The facial recognition software suffers from severe technical limitations and fails to perform reliably under real-world conditions.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The exclusion of Black asylum seekers was an unexpected and unintentional outcome of deploying the CBP One app's facial recognition system post-deployment.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The system is used in migration, asylum, and border control management, specifically involving biometric identification and facial recognition, which has significant implications for fundamental rights.

AI system and alleged parties

• AI system: CBP One facial recognition feature
• AI purpose: Face Recognition; Identification
• Behaviour type: Assistant
• Alleged developer: US Customs and Border Protection
• Alleged deployer: US Customs and Border Protection
• Alleged harmed parties: Haitian asylum seekers, Black asylum seekers, African asylum seekers

Harm severity

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Substantial
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Substantial
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Substantial, indirect Substantial
• Civil rights: direct Substantial, indirect Substantial
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Substantial, indirect Negligible
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Physical

Reported: The report does not explicitly describe direct physical injuries caused by the app itself, but notes that stranded migrants face violence.

Directly caused: N/A

Indirectly caused: The app's failure forces thousands of vulnerable migrants to remain in dangerous border camps where they are subjected to kidnappings, extortion, and violence by organized crime.

Inferred additional harm: It is highly likely that some of the thousands of stranded migrants suffered physical injuries or death due to cartel violence and harsh environmental conditions while waiting.

Financial

Reported: The report explicitly describes financial losses incurred by migrants trying to make the app work.

Directly caused: N/A

Indirectly caused: Migrants are forced to buy expensive $1,000 cellphones, pay for mobile data plans, or pay lawyers up to $7,000 for assistance in navigating the app.

Inferred additional harm: Widespread financial exploitation and resource depletion among thousands of impoverished migrants who must divert limited funds to bypass technical glitches.

Differential treatment

Reported: The report explicitly describes systematic differential treatment of individuals based on skin tone.

Directly caused: The app's facial recognition algorithm fails to register darker skin tones, meaning lighter-skinned migrants successfully get appointments while Black migrants are blocked.

Indirectly caused: The technical bias creates a tiered asylum system where access is determined by race, language, and economic status.

Inferred additional harm: Systemic discrimination against thousands of Black asylum seekers across all border entry points due to technological exclusion.

Civil rights

Reported: The report explicitly describes violations of the legal right to seek asylum.

Directly caused: The app's failure directly blocks Black migrants from submitting their photos, preventing them from exercising their legal right to request asylum.

Indirectly caused: The mandatory use of a flawed app creates impermissible barriers to asylum, violating international human rights law.

Inferred additional harm: Thousands of individuals are denied their fundamental human right to seek safe refuge due to technological exclusion.

Privacy

Reported: The report explicitly describes serious privacy concerns raised by requiring migrants to submit sensitive biometric and location data.

Directly caused: The app mandates the collection and submission of sensitive biometric data and precise geolocation data from vulnerable asylum seekers.

Indirectly caused: N/A

Inferred additional harm: Potential misuse or unauthorized sharing of sensitive personal data of thousands of migrants without adequate safeguards.

Psychological

Reported: The report explicitly describes severe distress, anxiety, and desperation among migrants unable to use the app.

Directly caused: N/A

Indirectly caused: Migrants are described as 'growing desperate' and facing a 'stark calculus' between buying food or pay-as-you-go data, causing significant mental distress.

Inferred additional harm: Widespread anxiety, trauma, and distress are highly likely among thousands of families who were split up or stranded in dangerous conditions due to the app's failures.

People affected

• Occurrences reported: 1
• People reportedly harmed: 5000
• People reportedly exposed: 5000

Potential causes

Management

• Failure to Consult NGOs: DHS did not consult civil society groups before the app rollout.
• Inadequate Risk Assessment: Management failed to address known facial recognition demographic bias.
• Poor Support Mechanisms: No assistance provided to migrants facing technical app rejections.

Technology

• Facial Recognition Bias: The algorithm fails to recognize and register darker skin tones.
• Frequent App Glitches: The app frequently crashes and displays error messages during use.
• Family Appointment Logic Flaws: The system splits families by failing to register children under six.

Data Inputs

• Biometric Photo Requirements: Mandatory photo submission relies on biased facial recognition tech.
• Poor Low-Light Image Input: The app requires high-quality photos that fail in poor lighting camps.
• Biometric Data Mandate: Forcing sensitive biometric data input creates technical bottlenecks.

Human Factors

• Lack of Smartphone Access: Migrants lack high-end phones or stable internet to run the app.
• Language Barriers: App was initially only in English and Spanish, excluding others.
• Physical Accessibility Issues: Users with cataracts or poor eyesight struggle to use the interface.

Process and Methods

• Mandatory App Usage Policy: DHS made the app the exclusive pathway for seeking asylum.
• Lack of Pre-Rollout Testing: The application was not sufficiently tested for glitches before rollout.
• No Alternative Process: Lack of offline or non-app alternatives for border processing.

Regulatory Environment

• Title 42 Restrictions: Strict border policies forced reliance on automated screening apps.
• Lack of Biometric Regulations: Absence of laws banning or restricting biased biometric technologies.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts from multiple independent non-profit organizations and legal advocates working directly with affected migrants at different border locations. The technical failure of the facial recognition software is clearly documented, along with its direct impact on Black asylum seekers.
• Ambiguities identified: The exact technical specifications of the facial recognition algorithm used in the CBP One app are not detailed in the reports.

The deployment of the CBP One app with biased facial recognition software systematically excluded darker-skinned asylum seekers from processing. While it presents negligible kinetic or intelligence threats, it represents a substantial human rights and operational challenge at the sovereign border due to algorithmic discrimination.

• Overall national security impact: Substantial
• Response level: Substantial
• Scope: Single nation
• Primary target: United States
• Other affected: Mexico, Haiti
• Alleged perpetrator: United States Customs and Border Protection

Threat characteristics

• Imminence: Long-term. Represents an ongoing systemic and operational challenge rather than an immediate, acute national security crisis.
• Autonomy: Human-supervised. The AI system automates the identity verification process, but operates under the broader administrative oversight of border officials.
• Novelty: Established threat. Racial and demographic bias in facial recognition technology is a well-documented and established technical limitation.

Impact by dimension

• Physical security: Minor. No direct kinetic threat, but the technical failure indirectly forces thousands of vulnerable migrants to remain stranded in dangerous border regions susceptible to violence.
• Information security: Negligible. No information warfare, intelligence compromise, or systematic disinformation campaigns are associated with this incident.
• Sovereignty: Minor. Disrupts standard border control and asylum processing operations, representing a minor disruption to government functions at the border.
• Economic security: Negligible. No significant threat to strategic industries, critical infrastructure, or national technological competitive advantage identified.
• Societal stability: Substantial. Systemic algorithmic bias in a mandatory government app results in unequal performance and exclusion of Black asylum seekers, impacting civil liberties and human rights at scale.