Reported Emergence of 'Vegetative Electron Microscopy' in Scientific Papers Traced to Purported AI Training Data Contamination

Researchers reportedly traced the appearance of the nonsensical phrase "vegetative electron microscopy" in scientific papers to contamination in AI training data. Testing indicated that large language models such as GPT-3, GPT-4, and Claude 3.5 may reproduce the term. The error allegedly originated from a digitization mistake that merged unrelated words during scanning, and a later translation error between Farsi and English.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The AI systems inadvertently generate and spread a nonsensical, incorrect scientific term, leading to its proliferation in academic literature and undermining scientific consensus.

Additional risk subdomains

• 7.3 Lack of capability or robustness: The models lack the robustness to identify and filter out obvious nonsense, reliably reproducing errors ingested from their training data.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The propagation of the nonsense term is driven by the post-deployment outputs of LLMs that unintentionally memorized training data errors.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The systems involved are general-purpose chatbots and generative AI models (GPT-3, GPT-4o, Claude 3.5) which fall under the category of AI systems that require transparency obligations so users know they are interacting with AI-generated content.

AI system and alleged parties

• AI system: GPT-3, GPT-4o (OpenAI)
• AI purpose: Writing Assistant; Technical Text Generation
• Behaviour type: Assistant
• Alleged developer: OpenAI, Anthropic
• Alleged deployer: Scientific authors, Researchers, OpenAI, Anthropic
• Alleged harmed parties: Scientific record, Scientific publishers, Scientific authors, Scholars, Researchers, Readers of scientific publications, Peer reviewers, Academic integrity

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Substantial, indirect Substantial
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Epistemic

Reported: The report explicitly describes epistemic harm through the pollution of scientific literature with a nonsense term generated or perpetuated by AI models.

Directly caused: AI models like GPT-3, GPT-4o, and Claude 3.5 consistently complete prompts with the nonsense term, embedding it in their knowledge bases and leading to its inclusion in at least 22 academic papers.

Indirectly caused: The proliferation of this term complicates the peer-review process, leads to journal retractions and corrections, and pollutes the scientific information ecosystem.

Inferred additional harm: It is highly likely that many other undiscovered nonsense terms and 'digital fossils' exist in AI systems, further degrading the reliability of AI-assisted scientific research.

People affected

• Occurrences reported: 1
• People reportedly exposed: 100

Potential causes

Management

• Publisher Inaction: Publishers responded inconsistently, sometimes defending the errors.
• Proprietary Data Secrecy: Tech companies hide training datasets, making errors hard to correct.

Technology

• LLM Error Reinforcement: Models like GPT-4o permanently embed and reproduce the nonsense term.
• Faulty OCR Digitization: Digitization process merged text across columns to create a phantom term.

Data Inputs

• Contaminated Training Data: CommonCrawl dataset contains scraped pages containing the error.
• Farsi Translation Error: Translation error in Iranian papers introduced the term to the web.

Human Factors

• Paper Mill Fraud: Fraudulent networks use AI to quickly generate and publish papers.
• Uncritical AI Reliance: Authors submit AI-generated text without verifying technical terms.

Process and Methods

• Inadequate Peer Review: Reviewers and editors let obvious nonsense terms pass into journals.
• Ineffective Screening Tools: Automated tools only detect known errors, missing new variations.

Regulatory Environment

• Lack of AI Training Standards: Absence of regulations for auditing and cleaning massive training data.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide detailed, empirical, and consistent accounts of how the nonsense term originated, how it was ingested by specific AI models, and how it has propagated into academic literature. The findings are backed by scientific sleuths and automated tools like the Problematic Paper Screener.
• Ambiguities identified: The exact proportion of the 22 papers that were generated by AI versus those written by humans making translation errors is not fully quantified.
• Alternative interpretations: The incident could be viewed purely as a human translation and academic fraud issue (paper mills), but the empirical evidence showing LLMs consistently generating the term supports the AI-centric classification.

AI models ingested and reproduced a historical translation error, 'vegetative electron microscopy', polluting academic literature. While it highlights vulnerabilities in AI training datasets and scientific research integrity, it poses negligible direct threats to national security.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Unknown
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents a slow-burning strategic concern regarding AI data integrity rather than an active, imminent crisis.
• Autonomy: Human-supervised. AI models generate the text autonomously, but human authors and peer reviewers retain ultimate control over publication.
• Novelty: Evolved capability. Demonstrates an evolved manifestation of AI hallucination where historical errors become permanently embedded across multiple commercial models.

Impact by dimension

• Physical security: Negligible. No physical safety threats, critical infrastructure manipulation, or kinetic impacts are associated with this data ingestion error.
• Information security: Negligible. The incident involves unintentional data propagation rather than a deliberate foreign disinformation campaign or intelligence compromise.
• Sovereignty: Negligible. No impact on state authority, electoral systems, or core government decision-making processes.
• Economic security: Minor. Highlights systemic vulnerabilities in AI training data pipelines and risks of polluting scientific knowledge bases, affecting technological research integrity.
• Societal stability: Negligible. No threats to civil liberties, societal stability, or human rights were reported.