Purportedly Hallucinated Software Packages with Potential Malware Reportedly Downloaded Thousands of Times by Developers

Large language models have reportedly hallucinated non-existent software package names, some of which were subsequently uploaded to public repositories and incorporated into real codebases. In one case, a package named huggingface-cli, which was purported to have been originally suggested by an AI model, was downloaded more than 15,000 times. This dynamic enables what security researchers have termed "slopsquatting," in which attackers register hallucinated package names and introduce potential malware into software supply chains.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The incident is primarily caused by the AI systems' lack of capability and robustness, specifically their tendency to hallucinate non-existent software package names when generating code.

Additional risk subdomains

• 2.2 AI system security vulnerabilities and attacks: The hallucinated packages create a security vulnerability in the software supply chain that can be exploited by malicious actors to distribute malware.
• 4.3 Fraud, scams, and targeted manipulation: Attackers can weaponize these hallucinations to conduct scams or distribute malware, and some have already used AI to generate realistic-sounding fake package names at scale.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The risk is caused by the AI system's unintentional hallucination of non-existent package names during post-deployment operation.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The systems involved are general-purpose AI chatbots and code generation assistants (like ChatGPT/GPT-4). According to the definitions, chatbots and general-purpose AI systems fall under Limited Risk, requiring transparency obligations so users know they are interacting with an AI.

AI system and alleged parties

• AI system: ChatGPT, GPT-4, unspecified (OpenAI)
• AI purpose: Code Generation; Technical Text Generation
• Behaviour type: Assistant
• Alleged developer: OpenAI, Meta, Google, DeepSeek AI, Cohere, BigScience
• Alleged deployer: Developers using AI-generated suggestions, Bar Lanyado
• Alleged harmed parties: Users downstream of software contaminated by hallucinated packages, Trust in open-source repositories and AI-assisted coding tools, Software ecosystems, Organizations that incorporated fake dependencies, Developers and businesses incorporating AI-suggested packages, Alibaba

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Epistemic

Reported: Yes, the report explicitly describes AI systems generating false information by hallucinating non-existent software package names.

Directly caused: AI models generated code recommending non-existent packages (e.g., 'huggingface-cli' via pip instead of the correct command), misleading developers.

Indirectly caused: Google's AI Overview parroted fake package READMEs, providing a false sense of legitimacy to fake packages.

Inferred additional harm: Widespread hallucination of package names across millions of code generations degrades the reliability of AI-assisted development environments.

People affected

• Occurrences reported: 1
• People reportedly exposed: 2000

Potential causes

Management

• Prioritizing Speed Over Security: Organizations rush AI integration without establishing safe coding guidelines.
• Inadequate AI Risk Assessments: Management fails to assess the supply chain risks of AI-generated code.
• No training for developers on AI: No training for developers on the risks of AI package hallucinations.

Technology

• Persistent LLM Hallucinations: Models repeatedly invent plausible but non-existent package names.
• Increased Scale Reduces Reliability: Larger models give plausible but wrong answers instead of admitting gaps.
• AI Overviews Parroting Fake Info: Search engines summarize and praise fake packages, giving false legitimacy.

Data Inputs

• Unverified Package Lists in Prompts: LLMs lack access to real-time verified package databases during generation.
• Repetitive Prompt Artifacts: Certain prompts consistently trigger the same hallucinated package names.
• Lack of Registry Reference Data: Models generate code without cross-referencing live package registries.

Human Factors

• Vibe Coding Blind Trust: Developers copy and paste AI suggestions without verifying if packages exist.
• Poor Human Evaluation of LLM Outputs: Humans fail to detect incorrect AI answers 10 to 40 percent of the time.
• Overreliance on AI Assistants: Developers assume AI-generated installation instructions are inherently safe.

Process and Methods

• Automatic Installation by AI Agents: AI agents install recommended packages directly without human intervention.
• Lack of Dependency Verification: Failure to use dependency scanners, lockfiles, or manual package checks.
• Permissive Registry Publishing: Public package registries allow immediate registration of hallucinated names.

Regulatory Environment

• Lack of LLM Provider Transparency: No regulatory mandate for AI providers to report package hallucination rates.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed, and technically sound explanations of the package hallucination vulnerability ('slopsquatting') across multiple security firms (Lasso Security, Socket) and academic studies. The specific case of Alibaba's GraphTranslator is well-documented, and the mechanics of the vulnerability are clearly explained.
• Ambiguities identified: None of significance; the distinction between benign research packages and actual malicious exploitation is clearly maintained.
• Alternative interpretations: The incident could be viewed purely as a human security issue (typosquatting/supply chain hygiene), but the central role of AI hallucinations in generating the predictable attack surface makes it primarily an AI safety/robustness issue.

Security researchers identified 'slopsquatting,' a software supply chain vulnerability where attackers register malicious packages matching AI-hallucinated names. This poses a substantial threat to global technological security by introducing vulnerabilities into major codebases, as demonstrated by Alibaba's GraphTranslator project.

• Overall national security impact: Substantial
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Global developer community
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing software supply chain vulnerability and strategic security concern rather than an immediate, acute crisis.
• Autonomy: Human-controlled. AI models act as coding assistants, generating suggestions that require human developers to accept, integrate, and execute.
• Novelty: Evolved capability. Adapts traditional typosquatting and supply chain attack methods to exploit the novel phenomenon of generative AI model hallucinations.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or kinetic targets were affected by the reported software package hallucinations.
• Information security: Minor. While creating a vector for malware distribution, the reported event was a benign test. No active intelligence compromise or state-sponsored information warfare was documented.
• Sovereignty: Negligible. The incident did not impact state authority, electoral systems, or core government administrative functions.
• Economic security: Substantial. The vulnerability compromises software supply chain integrity for major tech firms like Alibaba. Since AI assistants are widely used in strategic tech sectors, this creates a notable systemic risk to technological security.
• Societal stability: Negligible. No negative impacts on civil liberties, societal stability, or human rights were reported or observed.