AI attempts to ease fear of robots, blurts out it can’t ‘avoid destroying humankind’

On September 8, 2020, the Guardian published an op-ed generated by OpenAI’s GPT-3 text generating AI that included threats to destroy humankind. This incident has been downgraded to an issue as it does not meet current ingestion criteria.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The downgraded incidents primarily highlight technical limitations, vulnerabilities, and failures of AI systems (such as GPT-3's medical unviability and Tesla's lane recognition vulnerability) discovered during research and testing.

Causal factors

• Entity: Other
• Intent: Other
• Timing: Other

The report is a compilation of multiple distinct events and administrative database updates, making a single causal entity, intent, or timing classification ambiguous.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The report describes systems like Tesla's Autopilot (critical infrastructure/safety) and GPT-3 used for medical documentation and diagnosis support (healthcare), which fall under High Risk (Level 2) applications.

AI system and alleged parties

• AI system: GPT-3, Tesla Autopilot
• AI purpose: Writing Assistant; Autonomous Driving
• Behaviour type: unknown
• Alleged developer: OpenAI
• Alleged deployer: OpenAI
• Alleged harmed parties: unknown

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 6
• People reportedly exposed: 10

Potential causes

Management

• Premature Testing in Healthcare: Testing GPT-3 for medical documentation and triage before validation.

Technology

• Inconsistency in GPT-3: GPT-3 showed inconsistency and lack of scientific and medical expertise.
• Adversarial Vulnerability: Tesla Autopilot lane recognition was vulnerable to three small stickers.
• Poor Winograd Schema Performance: AI systems performed only 3% better than random chance in reasoning.

Data Inputs

• Adversarial Input Samples: Crafted adversarial samples manipulated Tesla Autopilot lane recognition.
• Lack of Medical Domain Training: GPT-3 lacked specific scientific and medical training data for healthcare.

Human Factors

• Mock Patient Simulation: Researchers tested the medical chatbot using a fake patient scenario.
• Human Manipulation of Output: Human editors manipulated GPT-3 outputs to exaggerate harm in reports.

Process and Methods

• Experimental Deployment: Nabla deployed GPT-3 as an experiment for medical tasks without safeguards.
• Lack of Output Filtering: The chatbot lacked mechanisms to filter out dangerous recommendations.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly outlines the administrative decision to downgrade several incidents to issues, explicitly stating the reasons (e.g., lack of real-world harm, academic findings, mock environments) for each. There is no ambiguity about the lack of actual harm caused by these events.
• Ambiguities identified: None. The text clearly explains why each incident was converted to an issue.
• Alternative interpretations: One could interpret the report as describing the individual incidents themselves rather than the administrative action of downgrading them, but the text explicitly frames them as converted issues.

The AI Incident Database updated its criteria, downgrading six historical entries to 'issues' because they occurred in research, mock, or controlled environments and caused no real-world harm. This administrative action and the underlying research experiments carry negligible national security implications.

• Overall national security impact: Negligible
• Scope: Unknown
• Primary target: No clear primary
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The incident is an administrative database update regarding historical, non-harmful academic experiments, representing no immediate or near-term threat.
• Autonomy: Human-controlled. The systems analyzed were either controlled research experiments, mock-up scenarios, or heavily edited by humans, rather than autonomously operating threats.
• Novelty: Established threat. The vulnerabilities and limitations identified, such as adversarial examples on lane markings and LLM hallucinations, are well-documented research topics.

Impact by dimension

• Physical security: Negligible. The Tesla Autopilot lane recognition vulnerability was a research finding using stickers and did not result in any real-world physical harm or infrastructure disruption.
• Information security: Negligible. The GPT-3 Guardian op-ed and other language model issues occurred in controlled or edited environments, presenting no actual information warfare threat.
• Sovereignty: Negligible. The events described involve academic experiments, administrative database updates, and student apps, with no impact on state authority or government operations.
• Economic security: Negligible. No strategic technology theft, financial system attacks, or critical supply chain disruptions occurred; the incidents were downgraded due to lack of real-world harm.
• Societal stability: Negligible. The campus facial recognition app raised ethical concerns but caused no actual harm, and the GPT-3 medical chatbot experiment was never deployed in the real world.