Transgender Uber Drivers Mistakenly Kicked off App for Appearance Change during Gender Transitions

Transgender Uber drivers reported being automatically deactivated from the app due to Real-Time ID Check failing to account for difference in appearance of people undergoing gender transitions.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.3 Unequal performance across groups

The facial recognition system performed with lower accuracy for transgender drivers undergoing transition, leading to automated deactivations and unequal access to the platform.

Additional risk subdomains

• 6.2 Increased inequality and decline in employment quality: The automated deactivations led to loss of income and increased barriers to employment for transgender drivers.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The deactivations were caused by the automated Real-Time ID Check AI system failing to recognize transgender drivers post-deployment, which was an unintentional outcome of its security goals.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The system is used for biometric identification (facial recognition) and as part of an employment/worker management platform, both of which are classified as High Risk under the EU AI Act.

AI system and alleged parties

• AI system: Microsoft Cognitive Services (Microsoft)
• AI purpose: Face Recognition; Workforce Monitoring and Evaluation
• Behaviour type: Autonomous
• Alleged developer: Uber
• Alleged deployer: Uber
• Alleged harmed parties: transgender Uber drivers

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Minor
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Differential treatment

Reported: The report explicitly describes differential treatment of transgender drivers caused directly by the incident.

Directly caused: Transgender drivers undergoing transition were repeatedly flagged and deactivated by the Real-Time ID Check system due to physical changes, unlike cisgender drivers.

Indirectly caused: Transgender drivers had to undertake extra labor, such as driving hours to support centers and verifying identity 100 times, to maintain platform access.

Inferred additional harm: Widespread systemic bias against transitioning individuals using automated facial recognition systems across the platform.

People affected

• Occurrences reported: 2
• People reportedly harmed: 2
• People reportedly exposed: 2

Potential causes

Management

• Prioritization of Automation: Uber prioritized automated efficiency over edge-case driver safety.
• Inadequate Risk Assessment: Management failed to assess risks of facial recognition for trans drivers.

Technology

• Inflexible Facial Recognition: Microsoft Cognitive Services failed to recognize transitioning faces.
• Algorithmic Red Flags: System flagged natural facial changes as potential fraud.

Data Inputs

• Static Reference Photos: System compared current selfies against outdated driver's licenses.
• Lack of Photo History Integration: Uploaded transition photos were not used to update the matching baseline.

Human Factors

• Physical Appearance Changes: Drivers undergoing gender transition naturally changed their appearance.

Process and Methods

• Overly Automated Deactivation: System automatically suspended accounts without manual review.
• Inconvenient Resolution Process: Drivers had to travel to physical support centers to resolve issues.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly details the AI system used (Microsoft Cognitive Services / Uber Real-Time ID Check), the specific failure mode (failing to match transitioning transgender drivers), and the resulting financial and operational harms. The accounts of the two drivers are consistent and supported by Uber's own statements.
• Ambiguities identified: The exact number of affected transgender drivers nationwide is not specified, and the exact dollar amount of lost wages is not quantified.
• Alternative interpretations: None. The causal link between the facial recognition failure and the driver deactivations is clear and acknowledged by Uber.

An automated identity verification system used by Uber and powered by Microsoft failed to recognize transgender drivers undergoing gender transition, leading to erroneous account deactivations. This incident highlights ongoing civil rights and algorithmic bias concerns within commercial AI systems, but carries negligible national security implications.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. This represents an ongoing, systemic challenge with algorithmic bias rather than an active national security crisis.
• Autonomy: Human-supervised. The AI system automatically deactivates accounts based on facial recognition mismatches, but human operators at support centers can intervene and override the decision.
• Novelty: Established threat. Algorithmic bias and unequal performance of facial recognition technologies across demographic groups are well-documented and established issues.

Impact by dimension

• Physical security: Negligible. The incident involves a commercial ride-sharing platform's identity verification system and has no impact on physical security or critical infrastructure.
• Information security: Negligible. No intelligence capabilities, classified information, or information warfare operations were involved or compromised.
• Sovereignty: Negligible. The incident is strictly commercial and does not affect state authority, electoral systems, or core government operations.
• Economic security: Negligible. While individual drivers suffered temporary financial losses, there is no threat to national economic stability, critical supply chains, or strategic technologies.
• Societal stability: Minor. The automated system exhibited algorithmic bias and unequal performance against transgender individuals, affecting their livelihoods, but the impact is localized and manageable through standard civil rights and corporate procedures.