Uber Autonomous Cars Running Red Lights

Uber vehicles equipped with technology allowing for autonomous driving running red lights in San Francisco street testing.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The incident was primarily caused by the AI system failing to perform reliably under normal driving conditions, specifically failing to recognize traffic lights.

Additional risk subdomains

• 6.5 Governance failure: Uber deployed the autonomous vehicles on public roads without obtaining the required DMV testing permits and in defiance of regulatory demands.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The safety risk was caused by the autonomous driving system failing to recognize traffic lights after being deployed on public roads.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The system is used in critical infrastructure (traffic management/road transport). The reports describe self-driving cars operating autonomously on public roads, which directly impacts public safety and transport infrastructure.

AI system and alleged parties

• AI system: Uber autonomous driving system
• AI purpose: Autonomous Driving; Navigation Assistant
• Behaviour type: Autonomous
• Alleged developer: Uber
• Alleged deployer: Uber
• Alleged harmed parties: pedestrians, Motorists

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Epistemic

Reported: The reports describe Uber initially blaming human error and suspending safety drivers, despite internal knowledge that the autonomous system was at fault.

Directly caused: Uber publicly misled the press and public by claiming the red-light violations were due to human error, when internal documents showed the mapping system failed to recognize the lights.

People affected

• Occurrences reported: 1
• People reportedly exposed: 10

Potential causes

Management

• Prioritizing Speed Over Safety: Management deployed vehicles without permits to accelerate public rollouts.
• Deceptive Public Communication: Management falsely blamed human error instead of admitting system failure.

Technology

• Mapping Program Failure: Mapping program failed to recognize six traffic lights in San Francisco.
• Inability to Navigate Bike Lanes: Autonomous system failed to safely merge and navigate near bike lanes.

Data Inputs

• Inaccurate Map Data: The system relied on maps that did not properly identify traffic signals.
• Lack of Sensor Integration: Sensor data failed to override incorrect mapping data for red lights.

Human Factors

• Safety Driver Inaction: The backup driver failed to intervene and apply brakes to prevent violations.

Process and Methods

• Skipping Permit Requirements: Uber bypassed DMV permit processes, avoiding mandatory safety reporting.
• Inadequate System Testing: Deploying autonomous vehicles on public streets with unresolved mapping bugs.

Regulatory Environment

• Lack of Strict Local Enforcement: Initial lack of immediate enforcement allowed unpermitted testing on streets.

Information quality

• Classification confidence: High
• Reason for confidence: Multiple independent news outlets confirmed the details of the incident, backed by internal Uber documents and employee statements that contradicted Ubers initial public claims.
• Alternative interpretations: Uber initially claimed human error was to blame, but this was later debunked by internal documents.

Uber deployed unauthorized autonomous vehicles in San Francisco in 2016, which committed safety violations such as running red lights due to mapping failures. While representing a challenge to local regulatory authority and public safety, the incident has negligible national security implications and is managed through standard regulatory and law enforcement procedures.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Uber

Threat characteristics

• Imminence: Long-term. The incident represents a historical, localized regulatory and safety issue rather than an active national security crisis.
• Autonomy: Human-supervised. The vehicles operated autonomously but carried safety drivers who were expected to intervene, although they failed to do so during the red-light violations.
• Novelty: Evolved capability. Represented an early, unauthorized public deployment of autonomous vehicles in a major metropolitan area, demonstrating evolving safety and regulatory challenges.

Impact by dimension

• Physical security: Minor. Autonomous vehicles ran red lights on public roads due to mapping failures, presenting a localized safety risk. However, there was no mass casualty potential, kinetic attack, or critical infrastructure compromise.
• Information security: Negligible. No foreign disinformation, intelligence compromise, or state-sponsored information warfare occurred. Uber misled the public regarding human error, but this remains a corporate public relations issue.
• Sovereignty: Minor. Uber deployed vehicles in defiance of state regulators and operated without required permits, representing a minor challenge to local government regulatory authority but not national sovereignty.
• Economic security: Minor. Uber faced regulatory action, vehicle registration revocation, and relocation costs, but this caused no strategic technology theft, financial system instability, or loss of national AI competitiveness.
• Societal stability: Negligible. The incident did not involve mass surveillance, systematic discrimination, civil unrest, or erosion of fundamental human rights.