Cruise Autonomous Car Blocked Fire Truck Responding to Emergency

A fire truck in San Francisco responding to a fire was blocked from passing a doubled-parked garbage truck by a self-driving Cruise car on the opposing lane which stayed put and did not reverse to clear the lane.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The Cruise AVs failed to perform reliably and safely when encountering rare or complex road scenarios (edge cases) such as active emergency scenes, demonstrating a lack of capability and robustness in critical situations.

Additional risk subdomains

• 6.5 Governance failure: The reports highlight concerns over inadequate regulatory oversight and data-sharing restrictions that limit public transparency regarding autonomous vehicle safety.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incidents were caused by the autonomous driving decisions of deployed Cruise vehicles operating on public streets, which unintentionally interfered with emergency response activities.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The AI system is deployed in critical infrastructure (road traffic management and transport) and directly impacts public safety and emergency response services, which fits the definition of High Risk under the EU AI Act.

AI system and alleged parties

• AI system: Cruise AV (Cruise)
• AI purpose: Autonomous Driving
• Behaviour type: Autonomous
• Alleged developer: Cruise
• Alleged deployer: Cruise
• Alleged harmed parties: San Francisco firefighters, San Francisco Fire Department

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 42
• People reportedly harmed: 2
• People reportedly exposed: 200

Potential causes

Management

• Prioritizing Commercial Launch: Cruise pushed for paid ride-hail expansion before resolving edge cases.
• Insufficient Safety Transparency: Management withheld safety and collision details from public scrutiny.

Technology

• Inability to Handle Corner Cases: AV software struggles with rare, unexpected road situations like blockages.
• Failure to Detect Emergency Scenes: AVs failed to recognize active fire scenes, driving over hoses and tape.
• No Autonomous Reversing Capability: Vehicle stayed put instead of reversing to let the fire truck pass.

Data Inputs

• Inability to Sense Downed Wires: Sensors failed to detect downed electrical wires, leading to entanglement.
• Lack of Real-Time Emergency Data: AV lacked data inputs to identify active emergency zones and fire hoses.
• Operating Without Headlights: Sensor input or software failed to trigger headlights at night.

Human Factors

• Remote Operator Delay: Remote assistance took time to resolve the block, delaying the fire truck.
• Lack of First Responder Training: First responders lacked adequate training to quickly interact with stuck AVs.

Process and Methods

• Defaulting to Safe Stop in Lane: AV software defaults to stopping in travel lanes rather than pulling over.
• Inadequate Communication Channels: Firefighters had to communicate via radio to operators, causing delays.

Regulatory Environment

• Lack of Public Safety Data: DMV settlement allowed companies to shield safety data as trade secrets.
• Premature Permit Expansion: Regulators pushed to expand permits despite unresolved emergency interferences.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts of multiple specific incidents where Cruise AVs blocked emergency responders, backed by official filings from San Francisco city officials and the SFFD. While exact casualty and damage numbers from the fires are not fully detailed, the occurrence of the AI failures themselves is well-documented.
• Ambiguities identified: The exact number of injuries and the monetary value of property damage resulting from the delayed fire response on April 5, 2022, are not specified.
• Alternative interpretations: None. The role of the AVs in blocking the emergency vehicles is undisputed, though Cruise notes their vehicle technically 'yielded' as programmed.

Autonomous vehicles operated by Cruise repeatedly interfered with local emergency response operations in San Francisco, blocking fire trucks and running over active fire hoses. While causing local public safety hazards and minor property damage, the national security impact remains minor and localized, manageable under standard municipal and state regulatory procedures.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing safety and integration challenge for autonomous vehicle deployments rather than an active national security crisis.
• Autonomy: Full autonomy. The vehicles operate and make driving decisions in real-time without a human driver inside.
• Novelty: Evolved capability. Represents a significant real-world scaling up of autonomous vehicle edge-case failures, building on earlier isolated AV incidents.

Impact by dimension

• Physical security: Minor. Localized disruptions to municipal emergency services (fire department) in San Francisco, managed through local regulatory and standard safety procedures.
• Information security: Negligible. No intelligence compromise, classified data theft, or information warfare elements present in this civilian AV deployment incident.
• Sovereignty: Negligible. Incident involves municipal traffic and safety regulation of commercial vehicles, with no threat to state authority or core federal government functions.
• Economic security: Negligible. No strategic technology theft, financial system attacks, or national economic manipulation; strictly a commercial product safety issue.
• Societal stability: Negligible. No mass surveillance, systemic discrimination, or threat to civil liberties; limited to localized public safety concerns.