Tesla's Autopilot Misidentified the Moon as Yellow Stop Light

Tesla's Autopilot was shown on video by its owner mistaking the moon for a yellow stop light, allegedly causing the vehicle to keep slowing down.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The incident is primarily characterized by the AI system's failure to perform robustly and reliably under real-world conditions, misclassifying common environmental objects like the moon and signs.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: The marketing of the system as 'Full Self-Driving' encourages users to rely on it, creating safety risks when they must suddenly intervene to correct critical system failures.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The safety risks and erratic driving behaviors were caused by the AI system's perception errors and unexpected braking actions after being deployed to consumers.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The AI system is used as a safety component in road vehicles, which has significant implications for public safety and is subject to strict regulatory requirements.

AI system and alleged parties

• AI system: Autopilot, Full Self-Driving (Tesla)
• AI purpose: Autonomous Driving; Navigation Assistant
• Behaviour type: Autonomous
• Alleged developer: Tesla
• Alleged deployer: Tesla
• Alleged harmed parties: Tesla drivers

Harm severity

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 5
• People reportedly exposed: 20

Potential causes

Management

• Premature Marketing Promises: CEO promises full autonomy while the system is still highly flawed.
• Public Deployment of Beta Software: Releasing unfinished safety-critical software to untrained public drivers.

Technology

• Flawed Object Classification: Deep learning models misidentify celestial bodies and signs as traffic lights.
• Lack of Spatial Depth Perception: System fails to recognize that the moon is extremely far away.
• Erratic Path Planning Algorithms: Software executes jerky turns and sudden, random braking maneuvers.

Data Inputs

• Ambiguous Visual Stimuli: System cannot distinguish 2D billboard signs or the moon from real signals.
• Unstructured Urban Environments: Chaotic streets with dense traffic, pedestrians, and construction zones.

Human Factors

• Driver Fatigue and Stress: Drivers must remain constantly on guard to prevent sudden system failures.
• Over-reliance on Human Intervention: Safety model assumes the driver can react instantly to erratic maneuvers.

Process and Methods

• Inadequate Edge Case Validation: Failing to test and resolve obvious environmental bugs before release.
• Reliance on Post-Release Patches: Fixing critical driving bugs only after they are reported by users.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide detailed, first-hand accounts of the AI system's behavior, backed by social media video evidence and professional journalistic testing. The system's failures are clearly documented without conflicting narratives.
• Ambiguities identified: None. The technical failures (misclassifying objects) and the resulting vehicle behaviors (unexpected braking) are clearly described.
• Alternative interpretations: None. The incidents are clearly software perception and control failures rather than driver error.

Tesla's FSD Beta exhibited perception failures, misidentifying objects like the moon as traffic signals and causing erratic braking. While presenting local road safety hazards, the incident carries negligible national security risk as it is a commercial product issue requiring human supervision and is manageable through standard regulatory frameworks.

• Overall national security impact: Minor
• Response level: Moderate
• Scope: Single nation
• Primary target: No clear primary
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. This represents an ongoing product safety and technology development issue rather than an active, imminent national security crisis.
• Autonomy: Human-supervised. The FSD system operates steering and braking autonomously but requires constant human monitoring and immediate intervention to override errors.
• Novelty: Established threat. Perception failures and edge-case classification errors in autonomous vehicle systems are well-documented, established engineering challenges.

Impact by dimension

• Physical security: Minor. Erratic driving behavior and unexpected braking of consumer vehicles on public highways pose localized physical safety risks, but are manageable through standard regulatory oversight and driver intervention.
• Information security: Negligible. No information warfare, intelligence compromise, or systematic manipulation of information is associated with these commercial autopilot perception errors.
• Sovereignty: Negligible. The incident involves commercial driver-assist software anomalies and does not impact state authority, electoral systems, or core government operations.
• Economic security: Negligible. While reflecting on Tesla's technology robustness, these commercial software bugs do not threaten national economic stability, strategic supply chains, or overall technological security.
• Societal stability: Negligible. The perception failures represent localized traffic safety concerns rather than systematic human rights violations, mass surveillance, or threats to societal stability.