A Collection of Tesla Autopilot-Involved Crashes

Multiple unrelated car accidents result in varying levels of harm have been occurred while a Tesla's autopilot was in use.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The primary failure stems from the Autopilot system's inability to reliably detect and respond to stationary objects and lane boundaries under varying conditions.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Drivers routinely over-relied on the system, engaging in distracted behaviors like watching movies or looking at phones due to the system's branding and marketing.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incidents were caused by the Autopilot AI system making erroneous steering and braking decisions after being deployed to the public.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The system is a safety component of a passenger vehicle operating in critical transport infrastructure, directly impacting physical safety and public interest.

AI system and alleged parties

• AI system: None named
• AI purpose: Autonomous Driving; Navigation Assistant
• Behaviour type: Autonomous
• Alleged developer: Tesla
• Alleged deployer: Tesla
• Alleged harmed parties: Motorists

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Substantial, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Minor, indirect Negligible
• Financial: direct Minor, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Physical

Reported: The report explicitly describes multiple fatalities and injuries caused by the crashes.

Directly caused: Two fatalities (Walter Huang and Joshua Brown) and several injuries, including a broken foot for a Utah driver, serious injuries for an Arizona driver, and minor injuries for a Massachusetts trooper, a Massachusetts driver, and a Mazda driver.

Inferred additional harm: We can infer that other crashes mentioned (e.g., Connecticut, North Carolina) may have caused minor unrecorded injuries, possibly 2-3 additional injuries.

Property

Reported: Yes, severe damage to multiple vehicles.

Directly caused: Multiple Tesla vehicles were completely destroyed. Several police cruisers, a fire truck, an ambulance, a Mazda 3, and an Audi A4 sustained significant damage.

Inferred additional harm: The total value of property damage across these 8+ crashes is estimated to be in the hundreds of thousands of dollars.

Financial

Reported: No specific total dollar amounts of financial loss are explicitly reported.

Directly caused: The reports do not explicitly quantify the direct financial losses from the crashes, but each destroyed Tesla and emergency vehicle represents tens of thousands of dollars in property loss.

Inferred additional harm: Significant financial losses from vehicle replacements, medical bills, legal fees, and emergency response costs, likely totaling over $1,000,000 across all occurrences.

Epistemic

Reported: Yes, the reports describe public confusion and misleading claims regarding the safety and capabilities of Autopilot.

Directly caused: Tesla's marketing of 'Full Self-Driving Hardware' and videos showing hands-free driving misled users into believing the system had more capability than it did, contributing to over-reliance.

Inferred additional harm: Widespread public misunderstanding of the limitations of Level 2 driver-assist systems, potentially leading other drivers to unsafely trust similar technologies.

People affected

• Occurrences reported: 8
• People reportedly harmed: 7
• People reportedly exposed: 22

Potential causes

Management

• Prioritizing Deployment Over Safety: Releasing features before perfecting driver engagement monitoring.
• Defensive Public Communication: Blaming drivers rather than addressing systemic automation limitations.
• Ignoring Regulatory Recommendations: Failure to officially respond to NTSB recommendations on driver engagement.

Technology

• Limited Sensor Capabilities: Sensors failed to distinguish white trailer or detect stationary vehicles.
• Inadequate Driver Monitoring: Torque sensor was easily defeated and failed to ensure driver attention.
• Flawed Lane Tracking: Autopilot steered into gore area/median instead of maintaining correct lane.

Data Inputs

• Radar False-Braking Filtering: System tuned out stationary objects to avoid unnecessary braking events.
• Lack of Real-time Hazard Data: Lack of updated road data regarding damaged infrastructure like attenuators.
• Inadequate Visual Data Processing: Computer vision failed to contrast white trailer against a bright sky.

Human Factors

• Driver Overreliance on Automation: Drivers assumed system was fully autonomous, leading to severe distraction.
• Delayed Human Intervention: Drivers failed to override Autopilot despite receiving multiple warnings.
• Tricking Steering Wheel Sensors: Drivers bypassed hands-on warnings using weights or other workarounds.

Process and Methods

• Testing Beta Software on Public Roads: Deploying unproven driver-assist features in public beta risks human lives.
• Inconsistent Safety Validation: Lack of rigorous testing under diverse, real-world edge cases.
• Misleading System Nomenclature: Naming system 'Autopilot' creates false sense of complete autonomy.

Regulatory Environment

• Lack of Autonomous Vehicle Standards: Absence of federal regulations for Level 2 driver-assist safety systems.
• Inadequate Oversight of Beta Deploys: Regulators allowed public beta testing without strict safety mandates.
• No Standardized Driver Monitoring: Regulators did not mandate robust eye-tracking systems for driver engagement.

Information quality

• Classification confidence: High
• Reason for confidence: The reports include official NTSB executive summaries, police statements, and detailed journalistic accounts of multiple crashes, providing high-quality factual evidence.
• Ambiguities identified: Conflicting claims between Tesla and the NTSB/drivers regarding whether warnings were sufficient and whether the system or driver was primarily at fault.
• Alternative interpretations: Tesla argues the incidents are purely human driver negligence, whereas regulators and safety advocates view them as system design failures.

A series of fatal and injury-producing traffic accidents in the US involving Tesla vehicles operating on Autopilot. The incidents stem from system limitations in detecting stationary objects and driver over-reliance. While raising significant transport safety and regulatory issues, the national security impact remains minor and manageable under standard domestic safety procedures.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: No clear primary
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing regulatory and safety challenge rather than an imminent national security crisis.
• Autonomy: Human-supervised. Tesla Autopilot is a Level 2 driver-assist system requiring active human supervision, though prone to driver over-reliance.
• Novelty: Established threat. Multiple similar traffic accidents involving driver-assist systems have occurred over several years.

Impact by dimension

• Physical security: Minor. Accidents damaged highway infrastructure and emergency vehicles, representing localized transportation safety failures rather than systemic threats to critical infrastructure.
• Information security: Negligible. No intelligence compromise, classified data theft, or information warfare operations occurred.
• Sovereignty: Negligible. Core government operations, electoral systems, and state sovereignty were unaffected.
• Economic security: Negligible. The incident involves individual consumer vehicle accidents and property damage, without threat to strategic industries or national economic security.
• Societal stability: Negligible. No mass surveillance, systematic discrimination, or threat to societal stability occurred.