Sleeping Driver on Tesla AutoPilot

A Tesla Model S remained on autopilot while being operated by a drunk, sleeping operator whose hands were not on the wheel. The police had to slow the car down by slowing in front of the vehicle to activate its 'driver assist' feature .

Risk classification

• Primary risk domain: 5 Human-Computer Interaction
• Primary risk subdomain: 5.1 Overreliance and unsafe use

The driver unsafely relied on the Level 2 Autopilot system to operate the vehicle while he was completely incapacitated by alcohol and asleep.

Additional risk subdomains

• 7.3 Lack of capability or robustness: The Autopilot system failed to reliably detect the driver's prolonged lack of input and did not automatically slow to a stop as designed.

Causal factors

• Entity: Human
• Intent: Unintentional
• Timing: Post-deployment

The safety risk was initiated by the human driver's decision to operate the vehicle while highly intoxicated and fall asleep, relying on the Autopilot system.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The system is used in critical infrastructure and transport management (autonomous driving/traffic safety), which has significant implications for public safety.

AI system and alleged parties

• AI system: Autopilot (Tesla)
• AI purpose: Autonomous Driving; Navigation Assistant
• Behaviour type: Autonomous
• Alleged developer: Tesla
• Alleged deployer: Tesla, Motorist
• Alleged harmed parties: Motorists

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 1
• People reportedly exposed: 5

Potential causes

Management

• Overselling Capabilities: Company leadership promoted the system as more capable than it actually was.
• Inadequate Safety Safeguards: Management failed to implement sufficient controls to prevent driver distraction.
• Rollout Prioritization: Deploying features rapidly without fully addressing driver monitoring flaws.

Technology

• Easily Bypassed Hands-on Sensor: Torque sensors can be fooled by weights or household items like oranges.
• Lack of Driver Gaze Monitoring: Unlike competitors, Tesla lacked infrared cameras to verify driver attention.
• Glitchy Lane Tracking: System can get confused by lane splits or hill crests, veering dangerously.

Data Inputs

• Inadequate Sensor Inputs: Radar and cameras failed to detect stationary fire trucks or white trailers.
• Undetected Emergency Vehicles: Neural net lacked training data to recognize stationary first responder vehicles.
• Lack of Direct Driver State Data: System relies on torque feedback rather than direct physiological status data.

Human Factors

• Driver Over-reliance: Users overestimate system capabilities, leading to distraction or sleeping.
• Deliberate System Abuse: Drivers bypass safety limits to perform dangerous stunts for social media.
• Drunk Driving Misuse: Intoxicated drivers engage Autopilot as a chauffeur, ignoring backup duty.

Process and Methods

• Misleading Product Naming: Names like 'Autopilot' imply full autonomy, misleading the general public.
• Inadequate Customer Education: Tesla did not sufficiently train or inform buyers about autopilot limits.
• Variable Alert Intervals: Alert timing was variable, sometimes allowing minutes without driver input.

Regulatory Environment

• Outdated DUI Laws: Traffic laws fail to clearly define responsibilities in semi-autonomous cars.
• No Standard Police Protocols: Police had to improvise a dangerous maneuver to stop the moving vehicle.
• Lack of Federal Oversight: Regulators did not mandate driver-monitoring standards for level 2 systems.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts of the primary incident on November 30, 2018, including statements from the California Highway Patrol and details of the driver's arrest. While there is some technical ambiguity about how the Autopilot system remained engaged without driver input, the overall sequence of events is well-documented across multiple reputable sources.
• Ambiguities identified: It remains unconfirmed by Tesla whether Autopilot was officially engaged, and it is unclear how the driver bypassed the steering wheel torque requirements while asleep.
• Alternative interpretations: The driver might have used a physical device to trick the steering wheel sensors, or the system might have experienced a sensor malfunction.

An intoxicated driver fell asleep while operating a Tesla Model S on Autopilot, requiring California Highway Patrol to safely intercept and stop the vehicle. While the incident underscores public safety risks and driver overreliance on consumer semi-autonomous systems, it holds negligible national security implications.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: No clear primary
• Alleged perpetrator: Individual

Threat characteristics

• Imminence: Long-term. Refers to ongoing safety concerns regarding consumer adoption of semi-autonomous systems rather than an active national security crisis.
• Autonomy: Human-supervised. The Tesla Autopilot system is a Level 2 semi-autonomous feature designed for human supervision, though the driver failed to maintain oversight.
• Novelty: Established threat. Driver overreliance on driver-assist technologies and driving under the influence are well-documented, established public safety issues.

Impact by dimension

• Physical security: Negligible. Local traffic incident involving an impaired driver in a semi-autonomous vehicle, safely resolved by law enforcement with no impact on critical infrastructure or national security.
• Information security: Negligible. No elements of information warfare, intelligence compromise, or systematic manipulation of information were present in this incident.
• Sovereignty: Negligible. The incident was a routine local law enforcement matter and posed no threat to state sovereignty, territorial control, or core government functions.
• Economic security: Negligible. No threat to financial systems, strategic technology theft, or national economic security was associated with this consumer driving incident.
• Societal stability: Negligible. This was an isolated case of individual driver misconduct and consumer technology misuse, with no broader implications for societal stability or human rights.