Tesla Vehicle Running on Self-Driving Mode Crashes on City Streets

A Tesla vehicle running in self-driving mode outside the operating conditions supported by the software crashed and injured the driver. Subsequently, the driver filed a lawsuit against Tesla and a jury found no damages were warranted.

Risk classification

• Primary risk domain: 5 Human-Computer Interaction
• Primary risk subdomain: 5.1 Overreliance and unsafe use

The driver over-relied on the Autopilot system, using it on a city street despite manual warnings and failing to maintain attentiveness, which led to the crash.

Additional risk subdomains

• 7.3 Lack of capability or robustness: The Autopilot system swerved into a curb, demonstrating limitations in its capability to navigate city streets safely.

Causal factors

• Entity: Human
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by human driver distraction and overreliance on a semi-autonomous driving system post-deployment, leading to an unintentional crash.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk (Level 2): The Autopilot system is a safety-critical component in road transport, which has significant implications for physical safety and public interest.

AI system and alleged parties

• AI system: Autopilot (Tesla)
• AI purpose: Autonomous Driving; Navigation Assistant
• Behaviour type: Assistant
• Alleged developer: Tesla
• Alleged deployer: Justine Hsu
• Alleged harmed parties: Justine Hsu

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Minor, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Minor, indirect Negligible
• Financial: direct Minor, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Physical

Reported: Yes, the report explicitly describes physical injuries to the driver.

Directly caused: The crash of the Model S while on Autopilot directly caused the driver to suffer a fractured jaw, knocked out teeth, and facial nerve damage due to violent airbag deployment.

Property

Reported: Yes, the Tesla Model S was damaged in the crash.

Directly caused: The Tesla Model S sustained damage to its front end and deployed its airbags during the collision with the curb.

Financial

Reported: Yes, the plaintiff sought over $3 million in damages for her injuries and losses, but was awarded zero.

Directly caused: The plaintiff suffered medical expenses and vehicle damage from the crash, though specific dollar amounts for these losses are not detailed.

Indirectly caused: Tesla's stock dropped by 8% when the incident was first reported in 2019, representing temporary market value loss.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 1

Potential causes

Management

• Aggressive Technology Rollout: Promoting advanced features before technology was fully autonomous.

Technology

• Autopilot Street Limitations: Software was not designed to handle city street navigation safely.
• Delayed Warning Alert: Warning to take control was issued less than a second before impact.
• Violent Airbag Deployment: Airbag deployed with excessive force, causing severe facial injuries.

Data Inputs

• Undetected Road Curb: System failed to detect the curb or restraint on the city street.

Human Factors

• Driver Distraction: Driver was distracted and failed to monitor the road continuously.
• System Over-reliance: Driver assumed the system was fully self-driving rather than an assist.
• Ignoring Manual Warnings: Driver activated Autopilot on a city street against manual warnings.

Process and Methods

• Ineffective Warning Delivery: Warning method failed to ensure timely driver intervention.

Regulatory Environment

• Insufficient Regulatory Oversight: Lack of clear standards and enforcement for driver-assist systems.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, consistent details about the crash, the injuries, the legal arguments, and the jury's verdict. There is high consensus across the sources regarding the facts of the trial and the nature of the Autopilot system's involvement.
• Ambiguities identified: There is minor ambiguity regarding the exact technical reason why the vehicle swerved into the curb, as Tesla denied any system failure and blamed driver distraction, while the plaintiff claimed design defects.
• Alternative interpretations: The incident could be interpreted as a technical failure of the Autopilot system (7.3) rather than user overreliance (5.1), but the jury's verdict strongly supports the overreliance/distraction interpretation.

A domestic civil lawsuit and traffic accident involving a Tesla Model S on Autopilot. The jury found driver distraction, not system failure, to be the cause. The incident is a consumer safety and liability issue with negligible national security implications.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: No clear primary
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The incident is a historical, resolved civil case with no ongoing immediate threat to national security.
• Autonomy: Human-supervised. The Tesla Autopilot system functions as a driver-assistant technology requiring continuous human supervision and intervention capability.
• Novelty: Established threat. Crashes involving semi-autonomous driver-assist systems and subsequent legal disputes are established occurrences.

Impact by dimension

• Physical security: Negligible. A single-vehicle consumer accident resulting in localized physical injury and minor property damage, with no impact on critical infrastructure or national physical security.
• Information security: Negligible. No involvement of classified systems, intelligence assets, or information warfare campaigns.
• Sovereignty: Negligible. Standard domestic civil litigation process with no disruption to government operations or sovereignty.
• Economic security: Negligible. Minor, temporary stock fluctuation for a single commercial entity, with no systemic threat to national economic or technological security.
• Societal stability: Negligible. Individual consumer safety incident and subsequent civil trial with no broader impact on civil liberties, societal stability, or human rights.