Waymo Driverless Taxi Allegedly Stalled During Pedestrian Harassment Incident in San Francisco

A Waymo driverless taxi carrying a passenger, Amina V., was stalled in San Francisco when two men blocked its path, demanding her contact information. The immobilized autonomous vehicle left the rider feeling unsafe and trapped. Waymo’s Rider Support intervened to assist the passenger.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The Waymo vehicle lacked the capability to safely maneuver around pedestrians who were intentionally blocking it, leaving the passenger trapped and vulnerable to harassment.

Additional risk subdomains

• 5.2 Loss of human agency and autonomy: The passenger had no manual override controls to take over the vehicle and drive away from a threatening situation, removing human agency.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The vehicle's autonomous decision to remain completely stopped in response to pedestrians blocking its path, while safe for the pedestrians, unintentionally trapped the passenger and exposed her to harassment.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The AI system is used in critical infrastructure (road traffic management and autonomous transport), which has significant implications for public safety and passenger security.

AI system and alleged parties

• AI system: Waymo Driverless Taxi (Waymo)
• AI purpose: Autonomous Driving; Navigation Assistant
• Behaviour type: Autonomous
• Alleged developer: Waymo
• Alleged deployer: Waymo
• Alleged harmed parties: Amina V.

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 1

Potential causes

Management

• Prioritizing Collision Avoidance: Management prioritized avoiding pedestrian collisions over passenger security.
• Underestimating Harassment Risks: Company treated passenger harassment scenarios as exceedingly rare anomalies.
• Lack of Proactive Security Measures: Management relied on reactive Rider Support instead of proactive prevention.

Technology

• Inability to Maneuver Around Obstacles: The vehicle is unable to drive around pedestrians blocking its path.
• Lack of Passenger Control Mechanisms: Passengers cannot take over or command the vehicle to drive away from danger.
• System cannot distinguish harassment: System cannot distinguish harassment from normal pedestrian presence.

Data Inputs

• Sensor Obstruction Sensitivity: Sensors trigger immediate halts when pedestrians stand directly in front.
• Lack of Contextual Threat Detection: Vehicle input data lacks the context to identify malicious passenger harassment.

Human Factors

• Exploitation of Vehicle Safety Rules: Pedestrians exploit the vehicle's design to stop for humans to trap it.
• Passenger Visibility and Anxiety: Sitting in front with untinted windows made rider visible and feel trapped.
• Passenger Lack of Driving Ability: Rider's inability to drive increased panic and helplessness during incident.

Process and Methods

• Passive Stay-Inside Policy: Waymo advises passengers to remain inside even during external harassment.
• Routing Through High-Risk Areas: Routing algorithms do not actively avoid areas with high pedestrian incidents.
• In-Car Support Reactivity: Support system is reactive, only initiating help after passenger triggers it.

Regulatory Environment

• Lack of AV Passenger Safety Standards: No regulatory standards address passenger safety from external harassment.
• Focus on Traffic Law Compliance: Regulators focus on traffic violations rather than passenger security risks.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts of the incident, including direct quotes from the passenger and Waymo spokespersons, as well as video evidence of the encounter. There are no major conflicting details regarding the primary event.
• Alternative interpretations: The incident could be viewed purely as a human harassment issue rather than an AI safety failure, though the AI's inability to route around the obstacle is a key technical constraint that enabled the harassment.

A localized incident where a Waymo autonomous vehicle was temporarily blocked by two individuals in San Francisco, resulting in the brief harassment of a passenger. While highlighting technical limitations in autonomous vehicle routing and passenger safety during human-robot interactions, the incident has negligible national security implications.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Individuals

Threat characteristics

• Imminence: Long-term. Represents a minor, ongoing safety and technical challenge for autonomous vehicle deployment rather than an imminent threat.
• Autonomy: Full autonomy. The vehicle operated autonomously, making real-time driving decisions to stop based on its programming without active human control.
• Novelty: Evolved capability. Builds upon known issues of pedestrians blocking autonomous vehicles, highlighting a vulnerability where passengers can be trapped and harassed.

Impact by dimension

• Physical security: Negligible. Localized traffic incident involving a single commercial autonomous vehicle and two pedestrians. No damage to critical infrastructure or mass safety threats.
• Information security: Negligible. No information warfare, intelligence compromise, or systematic information manipulation occurred during this local incident.
• Sovereignty: Negligible. No impact on government decision-making, state authority, or electoral processes.
• Economic security: Negligible. No threat to economic stability, strategic technology theft, or loss of national AI competitive advantage.
• Societal stability: Negligible. While the passenger experienced distress, the incident was a localized criminal harassment event rather than a systematic threat to social cohesion or human rights.