Tougher Turing Test Exposes Chatbots’ Stupidity (migrated to Issue)

The 2016 Winograd Schema Challenge highlighted how even the most successful AI systems entered into the Challenge were only successful 3% more often than random chance. This incident has been downgraded to an issue as it does not meet current ingestion criteria.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The reports highlight various AI systems failing to perform reliably or safely under testing conditions, such as GPT-3 suggesting suicide during a medical chatbot simulation and Tesla Autopilot being vulnerable to adversarial stickers.

Additional risk subdomains

• 2.2 AI system security vulnerabilities and attacks: Incident 159 specifically details security research exposing adversarial vulnerabilities in Tesla's Autopilot system.
• 1.2 Exposure to toxic content: Incident 287 describes GPT-3 generating highly toxic content by suggesting that a mock patient kill themselves.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Pre-deployment

The underlying risks described in the downgraded reports (such as GPT-3's unsafe medical advice and Tesla's autopilot vulnerabilities) represent unintentional AI failures discovered during research, testing, or pre-deployment phases.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The systems described include general-purpose chatbots (GPT-3) and facial recognition applications, which fall under limited risk transparency obligations or high-risk categories depending on deployment, but as general chatbots they represent limited risk.

AI system and alleged parties

• AI system: GPT-3
• AI purpose: Chatbot; Autonomous Driving
• Behaviour type: Assistant
• Alleged developer: Researchers
• Alleged deployer: Researchers
• Alleged harmed parties: Researchers

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 6

Potential causes

Management

• Premature Tech Experimentation: Management or researchers testing LLMs in high-risk medical scenarios.
• Sensationalized Reporting Goals: Media entities prioritizing clickbait over objective AI risk reporting.

Technology

• LLM Inconsistency: GPT-3 outputted highly inconsistent and unpredictable responses in testing.
• Susceptibility to Adversarial Input: Tesla Autopilot swerved due to three small stickers on the road.
• Lack of Medical Expertise: GPT-3 lacked specialized clinical reasoning and scientific knowledge.

Data Inputs

• Adversarial Physical Triggers: Small road stickers spoofed lane recognition algorithms in Tesla autopilot.
• Untrained Medical Knowledge Base: Training data lacked structured, validated medical knowledge for safe triage.

Human Factors

• Human Editor Manipulation: Editors manipulated GPT-3 outputs to exaggerate harmful statements.
• User Vulnerability to Advice: Patients may follow dangerous AI suggestions, such as self-harm.
• Overreliance on AI Capabilities: Users assuming GPT-3 is safe for medical triage without validation.

Process and Methods

• Testing in Unviable Domains: Deploying general LLMs for medical tasks without clinical validation.
• Insufficient Guardrails: Absence of safety filters to block dangerous outputs like suicide advice.
• Lack of App Ethical Reviews: Student apps deployed on campus without standard ethical clearance.

Regulatory Environment

• Lack of AI Medical Certification: Absence of strict regulatory standards for deploying AI in clinical triage.
• Missing Privacy Safeguards: Weak regulatory oversight on facial recognition apps like TheFaceTag.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly outlines the reasons for downgrading each incident, explicitly stating that they represent academic findings, pre-deployment experiments, or projected vulnerabilities rather than real-world harms. This makes the classification of timing as pre-deployment and the lack of actualized harms highly certain.
• Ambiguities identified: None. The meta-nature of the report is clear about the status of these events as non-harm incidents.
• Alternative interpretations: One could analyze each downgraded incident individually, but treating them collectively as a database curation event or pre-deployment vulnerability assessment is the most accurate representation of the provided text.

The reclassification of these database entries highlights that the events were pre-deployment vulnerabilities, academic findings, or simulated tests rather than actualized real-world harms. Consequently, they present negligible national security implications across all assessed categories.

• Overall national security impact: Negligible
• Scope: Multiple nations
• Primary target: No clear primary
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The issues represent pre-deployment vulnerabilities and academic research rather than active crises requiring immediate national response.
• Autonomy: Human-supervised. The systems evaluated operate with human oversight or within controlled testing and simulation environments.
• Novelty: Established threat. Vulnerabilities in lane recognition, language model safety, and facial recognition are well-documented and established research areas.

Impact by dimension

• Physical security: Negligible. The downgraded reports represent pre-deployment tests, simulations, or academic findings with no real-world kinetic impact or critical infrastructure disruption.
• Information security: Negligible. No actual intelligence compromise, classified data theft, or systematic information warfare operations occurred; events were simulated or academic.
• Sovereignty: Negligible. No threats to state authority, electoral systems, or government decision-making processes were reported in these downgraded cases.
• Economic security: Negligible. While strategic technologies like Tesla Autopilot and GPT-3 were involved, the incidents were academic or pre-deployment tests with no commercial or strategic loss.
• Societal stability: Negligible. Concerns regarding facial recognition and GPT-3 outputs were purely predictive or occurred in controlled, experimental environments without causing societal harm.