Black Uber Eats Driver Allegedly Subjected to Excessive Photo Checks and Dismissed via FRT Results

A lawsuit by a former Uber Eats delivery driver alleged the company to have wrongfully dismissed him due to frequent false mismatches of his verification selfies, and discriminated against him via excessive verification checks.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.3 Unequal performance across groups

The facial recognition software exhibited unequal performance across demographic groups, specifically failing to accurately identify Black drivers, leading to their unfair dismissal.

Additional risk subdomains

• 6.2 Increased inequality and decline in employment quality: The use of biased automated systems for workforce management directly led to the loss of livelihood and degraded employment quality for minority gig workers.
• 7.4 Lack of transparency or interpretability: The lack of clarity and transparency regarding Uber's automated deactivation decisions made it difficult for the affected drivers to appeal or understand their dismissal.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was triggered by the Real-Time ID Check AI system failing to correctly verify the driver's identity due to algorithmic bias, which was an unintended outcome of its deployment.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The system is used for employment and worker management, specifically for identity verification and access control affecting a worker's livelihood, which falls under the High Risk category.

AI system and alleged parties

• AI system: Uber Eats Real-Time ID Check
• AI purpose: Identification; Workforce Monitoring and Evaluation
• Behaviour type: Tool
• Alleged developer: Uber Eats
• Alleged deployer: Uber Eats
• Alleged harmed parties: Uber Eats Black delivery drivers, Pa Edrissa Manjang

Harm severity

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Differential treatment

Reported: Yes, the report explicitly describes differential treatment based on race.

Directly caused: Pa Edrissa Manjang was subjected to frequent and excessive facial verification checks and was ultimately dismissed due to false mismatches, which he alleges was due to racial bias.

Indirectly caused: N/A

Inferred additional harm: Other Black and ethnic minority couriers are likely subjected to disproportionately higher rates of verification checks and false mismatches compared to white couriers.

Civil rights

Reported: Yes, the report explicitly describes violations of civil rights.

Directly caused: The drivers experienced a violation of their right to non-discrimination in the workplace due to the alleged use of racially biased software.

Indirectly caused: N/A

Inferred additional harm: The systemic deployment of biased biometric verification tools represents a broader threat to the civil rights of minority workers across the gig economy.

Privacy

Reported: Yes, the report describes the collection of sensitive biometric data.

Directly caused: Drivers were required to submit biometric facial scans (selfies) multiple times a day to verify identity.

Indirectly caused: N/A

Inferred additional harm: The continuous collection and processing of biometric data without robust safeguards or transparent processes compromises worker privacy.

People affected

• Occurrences reported: 2
• People reportedly harmed: 2
• People reportedly exposed: 2

Potential causes

Management

• Growth over worker welfare: Treating couriers as numbers rather than human beings.
• Poor algorithmic risk review: Deploying tech without auditing for racial discrimination risks.

Technology

• Racially biased algorithms: Facial recognition is less accurate for ethnic minorities.
• Frequent mismatch errors: System repeatedly failed to recognize legitimate couriers.

Data Inputs

• Skewed training datasets: Training data skewed towards white men, causing bias.

Human Factors

• Lack of human oversight: Support staff failed to manually verify courier photos.

Process and Methods

• Flawed deactivation process: Accounts were deactivated automatically based on mismatch flags.
• Opaque feedback channels: Couriers were not given clear reasons or verification methods.

Regulatory Environment

• Weak AI employment laws: Lack of regulatory standards for AI use in worker management.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, consistent details about the driver's dismissal, the technology used (Uber's Real-Time ID Check), and the legal proceedings. While Uber disputes that the automated system was the sole cause of dismissal, the core allegations of racial bias in the facial recognition tool are well-documented and supported by union statements and tribunal decisions.
• Ambiguities identified: Uber claims that automated facial verification was not the sole reason for dismissal and that human review is involved, creating some ambiguity about the exact level of automation in the deactivation decision.
• Alternative interpretations: The dismissal could be interpreted as a human management decision supported by a tool, rather than a purely automated AI decision.

A commercial dispute in the United Kingdom involving allegations of racial bias in Uber Eats' facial recognition identity verification tool. While raising important questions about civil rights, algorithmic discrimination, and gig-economy labor standards, the incident has negligible national security implications and is being addressed through standard legal channels.

• Overall national security impact: Minor
• Response level: Moderate
• Scope: Single nation
• Primary target: United Kingdom
• Alleged perpetrator: Uber

Threat characteristics

• Imminence: Long-term. The incident represents an ongoing regulatory and ethical concern regarding AI bias rather than an active national security crisis.
• Autonomy: Human-supervised. The AI system performed automated identity verification, though Uber asserts that human review processes were integrated into the final deactivation decisions.
• Novelty: Established threat. Racial bias and unequal performance across demographics in facial recognition software is a well-documented and established technological issue.

Impact by dimension

• Physical security: Negligible. The incident involves a commercial food delivery application and has no impact on physical systems, critical infrastructure, or human safety.
• Information security: Negligible. No intelligence compromise, classified data theft, or information warfare operations are associated with this commercial labor dispute.
• Sovereignty: Negligible. This is a private employment dispute that does not threaten state authority, government decision-making, or constitutional processes.
• Economic security: Negligible. While the incident caused localized financial loss for the affected gig workers, it poses no threat to strategic industries or national economic security.
• Societal stability: Minor. The incident highlights concerns regarding algorithmic bias and civil rights in commercial biometric systems, but the impact is localized and manageable through standard legal tribunals.