Names Linked to Defamation Lawsuits Reportedly Spur Filtering Errors in ChatGPT's Name Recognition

ChatGPT has reportedly been experiencing errors and service disruptions caused by hard-coded filters designed to prevent it from producing potentially harmful or defamatory content about certain individuals by blocking prompts containing specific names, likely related to post-training interventions. The reported names are Brian Hood, Jonathan Turley, Jonathan Zittrain, David Faber, David Mayer, and Guido Scorza.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The AI system failed to perform reliably, crashing and terminating sessions when encountering specific names, and proved vulnerable to adversarial prompt injections exploiting these hard-coded rules.

Additional risk subdomains

• 3.1 False or misleading information: The underlying cause for the filters was the AI system's tendency to generate false, defamatory hallucinations about real individuals.
• 7.4 Lack of transparency or interpretability: OpenAI implemented hidden, blunt blocklists without transparency, leaving users and affected individuals confused about why certain names broke the system.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The service disruptions were caused by a deliberate human engineering decision to implement hard-coded, rule-based filters to block specific names post-deployment.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: ChatGPT is a chatbot, which falls under Risk Level 3 (Limited Risk) requiring specific transparency obligations so users know they are interacting with an AI.

AI system and alleged parties

• AI system: ChatGPT (OpenAI)
• AI purpose: Chatbot; Question Answering
• Behaviour type: Assistant
• Alleged developer: OpenAI, ChatGPT
• Alleged deployer: OpenAI, ChatGPT users
• Alleged harmed parties: Jonathan Zittrain, Jonathan Turley, Guido Scorza, David Mayer, David Faber, ChatGPT users, Brian Hood

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Differential treatment

Reported: The report explicitly describes differential treatment where individuals sharing the blocked names face service refusal when using the AI.

Directly caused: Users sharing the blocked names, or teachers trying to process lists containing those names, experienced service refusal as ChatGPT would terminate the session.

Psychological

Reported: The report mentions that being falsely accused of crimes can be 'chilling' for individuals, but does not explicitly describe clinical psychological harm.

Indirectly caused: The false accusations of sexual harassment, bribery, and embezzlement generated by ChatGPT prior to the blocks likely caused significant distress and reputational anxiety for the targeted individuals.

Epistemic

Reported: The report explicitly describes epistemic harm in the form of ChatGPT fabricating false, defamatory claims about individuals prior to the implementation of the filters.

Directly caused: ChatGPT fabricated false claims accusing Brian Hood of bribery, Jonathan Turley of sexual harassment, and Mark Walters of embezzlement.

People affected

• Occurrences reported: 1
• People reportedly harmed: 7
• People reportedly exposed: 7

Potential causes

Management

• Lack of System Transparency: Failure to communicate block lists and moderation reasons to users.
• Reactive Safety Measures: Implementing hasty blocks in response to legal threats.

Technology

• Blunt Rule-Based Filters: Hard-coded 'if-then' rules used to override LLM outputs abruptly.
• Black Box Neural Networks: Lack of understanding of LLM inner workings forces reliance on blunt rules.
• LLM Hallucinations: AI generating false, defamatory claims about real people.

Data Inputs

• Insufficient Training Data: Lack of accurate info about individuals leads to AI confabulations.
• Name Ambiguity in Data: Failure to distinguish between different people sharing the same name.

Human Factors

• Adversarial Prompt Injection: Users exploiting the filter using hidden text or visual prompt injections.

Process and Methods

• Inadequate Testing of Tools: Automated tools mistakenly flagged benign names as privacy violations.
• Blunt Data Redaction Methods: Using complete name bans instead of precise information filtering.

Regulatory Environment

• Strict Defamation Liabilities: Lawsuits forced rapid, poorly-tested filtering of specific names.
• Right to be Forgotten Laws: EU privacy regulations pressure platforms to remove personal data quickly.

Information quality

• Classification confidence: High
• Reason for confidence: The reports from reputable sources (Ars Technica, WSJ, NYT) provide consistent, detailed accounts of the name-blocking behavior, the specific names involved, the reasons behind the blocks (defamation concerns), and OpenAI's response confirming the glitch.
• Ambiguities identified: The exact list of all blocked names and the precise technical mechanism of the 'privacy tool' that mistakenly flagged David Mayer remain somewhat opaque.

OpenAI implemented hard-coded filters in ChatGPT to block specific names to prevent defamation lawsuits, leading to service disruptions and vulnerability to prompt injections. This incident represents a commercial software robustness and safety issue with negligible national security implications.

• Overall national security impact: Negligible
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Unknown
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The issue represents an ongoing technical challenge and vulnerability management concern for AI developers rather than an active national security crisis.
• Autonomy: Human-supervised. The AI system operates autonomously to generate text but is restricted by human-implemented rules, filters, and oversight mechanisms.
• Novelty: Established threat. AI hallucinations, prompt injections, and the use of hard-coded blocklists are well-documented and established issues in AI deployment.

Impact by dimension

• Physical security: Negligible. The incident involves software filter disruptions in a commercial chatbot and has no physical security or critical infrastructure implications.
• Information security: Negligible. No classified information, intelligence capabilities, or systematic information warfare campaigns were involved in this incident.
• Sovereignty: Negligible. While some affected individuals hold local public office, the incident does not threaten state authority, elections, or core government operations.
• Economic security: Negligible. The disruption represents a technical limitation of a commercial AI product and does not threaten strategic industries or national economic stability.
• Societal stability: Negligible. The incident caused minor service refusals and localized reputational concerns due to model hallucinations, but poses no threat to broad societal stability.