Sudden Braking by Tesla Allegedly on Self-Driving Mode Caused Multi-Car Pileup in Tunnel

A Tesla driver alleged Full Self Driving (FSD) braking unexpectedly as the cause for an eight-car pileup in San Francisco which led to minor injuries of nine people.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The primary failure was the Tesla FSD system's lack of robustness, specifically executing an unexpected 'phantom braking' maneuver on a highway, which represents a system-level capability failure.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: The reports highlight the 'vigilance problem' where drivers overrely on Level 2 systems and fail to intervene when the system malfunctions.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by the Tesla FSD software unexpectedly executing a braking maneuver post-deployment, which was an unintentional and unsafe action.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The system is used in critical infrastructure (traffic management/road safety) and transport, where failures pose significant implications for safety and human life. The reports describe a 'Full Self-Driving' driver-assist system operating on public highways, which falls under safety components of vehicles subject to third-party assessment or critical transport infrastructure.

AI system and alleged parties

• AI system: Full Self-Driving Beta (Tesla)
• AI purpose: Autonomous Driving; Navigation Assistant
• Behaviour type: Autonomous
• Alleged developer: Tesla
• Alleged deployer: Tesla
• Alleged harmed parties: traffic participants, Tesla drivers

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Minor, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Minor, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Physical

Reported: Yes, the reports explicitly describe physical injuries to nine people, including a two-year-old child.

Directly caused: Nine people were treated for minor injuries, including a two-year-old child who suffered a head abrasion and bruising, and two juveniles who were hospitalized.

Property

Reported: Yes, the reports describe an eight-vehicle crash, which directly caused damage to the vehicles involved.

Directly caused: Eight vehicles, including the Tesla Model S, were involved in a chain-reaction crash, resulting in physical damage to all of them.

People affected

• Occurrences reported: 1
• People reportedly harmed: 9
• People reportedly exposed: 9

Potential causes

Management

• Prioritizing Market Dominance: Releasing FSD to all users to differentiate from competitors.
• Disabling Driver Alerts: Management agreed to disable steering wheel nags for certain users.
• Disbanded PR Department: Lack of transparent communication with public regarding safety.

Technology

• Phantom Braking Bug: Software triggers sudden, unexpected braking without any obstacles present.
• Erratic Lane Change Action: FSD system initiated an unsafe lane change before stopping abruptly.
• Inadequate Object Detection: Autopilot systems struggle to detect stopped emergency vehicles with lights.

Data Inputs

• False Positive Obstacle Data: Sensor noise or misclassification triggers emergency braking sequences.
• Lack of Emergency Light Data: System fails to properly interpret flashing hazard lights of parked SUVs.
• Sensor Suite Limitations: Camera-only inputs may struggle with depth perception in tunnels.

Human Factors

• The Vigilance Problem: Humans struggle to monitor automated systems and fail to intervene in time.
• Overreliance on Automation: Misleading system names lead drivers to assume the car is fully autonomous.
• Delayed Driver Intervention: Driver failed to manually override the vehicle's unexpected braking.

Process and Methods

• Uncontrolled Beta Release: FSD Beta was opened to all users regardless of safety scores.
• Misleading System Branding: Marketing Level 2 tech as 'Full Self-Driving' misleads consumers.
• Deficient Driver Monitoring: Inadequate steering wheel nags allow drivers to disengage from driving.

Regulatory Environment

• Lack of Federal Restrictions: No federal rules govern testing of semi-autonomous vehicles on public roads.
• Delayed Regulatory Action: NHTSA has investigated crashes for years without taking enforcement action.
• Inadequate Safety Standards: Absence of standardized safety benchmarks for automated driving systems.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts of the crash, including police reports, surveillance video, and official NHTSA data confirming FSD engagement. There is high certainty regarding the sequence of events and the resulting harms.
• Ambiguities identified: None that significantly affect classification; while police initially could not verify FSD status, subsequent federal data confirmed it was active within 30 seconds of the crash.
• Alternative interpretations: One could argue the driver was solely at fault due to lack of vigilance, but the physical action causing the crash was initiated by the AI system's phantom braking.

A Tesla Model S operating on Full Self-Driving Beta software executed an unexpected phantom braking maneuver on the San Francisco Bay Bridge, causing an eight-vehicle pileup and nine minor injuries. The incident represents a minor physical safety and critical infrastructure concern, manageable within standard emergency response frameworks, with negligible broader national security implications.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: No clear primary
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The immediate traffic incident is resolved, but the systemic risk of autonomous vehicle safety and phantom braking remains an ongoing strategic concern.
• Autonomy: Human-supervised. The Tesla FSD software operated autonomously to steer and brake, but the system is designed to require active human supervision and intervention capability.
• Novelty: Established threat. Phantom braking and driver-assist system failures are established issues that have been subject to prior complaints and regulatory investigations.

Impact by dimension

• Physical security: Minor. The incident temporarily blocked a major transportation artery (San Francisco Bay Bridge) and caused an eight-vehicle pileup with minor injuries. While affecting critical transport infrastructure, it was manageable within standard emergency and traffic safety procedures.
• Information security: Negligible. No information warfare, intelligence compromise, or systematic information manipulation occurred in connection with this incident.
• Sovereignty: Negligible. This was a consumer vehicle accident and did not threaten state authority, territorial control, or core government operations.
• Economic security: Negligible. While indicating potential reliability issues in consumer autonomous tech, the incident did not threaten national economic stability, strategic industries, or technological competitive advantage.
• Societal stability: Negligible. The incident does not represent a threat to social cohesion, civil liberties, or population-scale safety, remaining a localized transportation safety issue.