10.2 Package Hallucination and Slopsquatting¶

Section 6.6 introduced slopsquatting—a term coined by Seth Larson, security developer-in-residence at the Python Software Foundation—as an emerging attack vector where adversaries register package names that AI coding assistants commonly hallucinate. This section examines the phenomenon in greater technical depth, exploring why AI hallucinations occur, what makes them exploitable, and how organizations can defend against this attack in their AI-assisted development workflows.

The core vulnerability is straightforward: AI models generate plausible-sounding package names that don't exist, attackers register those names with malicious packages, and developers who trust AI suggestions install the attackers' code. What makes this attack particularly concerning is that hallucinations are not random—they're repeatable and predictable, enabling systematic exploitation.

The Hallucination Phenomenon¶

Large language models (LLMs) powering coding assistants generate text by predicting the most likely next tokens based on patterns in their training data. When generating import statements or dependency specifications, models produce package names that seem probable given the context—but "probable" doesn't mean "real."

Why Hallucinations Occur¶

Several factors contribute to package name hallucinations:

Pattern completion over memory: Models complete patterns rather than retrieving exact facts. A model might generate flask-security-utils because "flask," "security," and "utils" commonly appear together, not because it "remembers" a specific package.

Training data noise: Training corpora include:

• Hypothetical examples in documentation
• Packages that existed but were removed
• Typos and errors in existing code
• Fictional examples in tutorials

Ecosystem fragmentation: Package ecosystems contain hundreds of thousands of packages. No model maintains a complete, accurate registry.

Naming convention inference: Models learn naming patterns (e.g., python-*, py*, *-utils) and generate novel combinations that fit patterns but don't correspond to real packages.

Temporal misalignment: Models trained on 2022 data may reference packages removed, renamed, or subsumed since then.

The Appearance of Validity¶

Hallucinated package names aren't random strings—they follow ecosystem conventions:

• Appropriate prefixes (py-, node-, @scope/)
• Conventional structure (kebab-case, lowercase)
• Semantically meaningful names that fit the programming context
• Plausible version numbers when versions are included

This plausibility is precisely what makes them dangerous. A developer seeing import flask_authentication_helper in AI-generated code has no immediate reason to suspect the package doesn't exist.

Research Quantification¶

Academic research has begun quantifying the scope of package hallucination.

The Spracklen Study:

Research by Spracklen et al. ("We Have a Package for You!") systematically tested AI models for package hallucination across multiple ecosystems:

Key Findings:

• Approximately 19.7% of AI-generated code samples overall referenced packages that don't exist in their target registries
• Hallucination rates varied significantly by model type
• The study tested Python (PyPI) and JavaScript (npm) ecosystems, which showed the highest representation in training data
• Commercial models (GPT-4, Claude, etc.) hallucinated less than open-source alternatives

Endor Labs' 2025 State of Dependency Management report¹ reinforced these findings with broader implications: only 1 in 5 dependency versions recommended by AI coding assistants were safe to use, containing neither hallucinations nor known vulnerabilities. The report notes that as developers increasingly rely on AI tools to accelerate coding, these agents "not only produce insecure code but also import vulnerable or non-existent open-source dependencies at scale."

Model Type Variation:

The study focused on Python (PyPI) and JavaScript (npm), generating 576,000 code samples to measure hallucination patterns across different AI models.

Corroborating Research:

A follow-up study by Krishna et al.² ("Importing Phantoms," January 2025) extended these findings to 11 models across Python, JavaScript, and Rust. Hallucination rates ranged from 0.22% to 46.15% depending on the model, with Rust showing the highest average rate at 24.74%, followed by Python at 23.14% and JavaScript at 14.73%. The study revealed two counterintuitive findings: coding-specialized models hallucinated more than general-purpose models, and induced hallucinations (where prompts specifically reference non-existent packages) occurred at nearly double the rate of natural hallucinations. The researchers also found a strong inverse correlation between a model's hallucination rate and its performance on standard coding benchmarks like HumanEval (p = -0.7887), suggesting that models that write better code also hallucinate fewer packages.

Repeatability: Why Attacks Are Viable¶

Random hallucinations would be difficult to exploit. An attacker would need extraordinary luck for their registered package name to match what an AI happens to generate for a specific developer. But hallucinations are not random—they're repeatable.

Spracklen's Repeatability Finding:

The research found⁴ a bimodal pattern in hallucination behavior:

• 43% of hallucinated packages were regenerated in all 10 iterations of the same prompt
• 58% appeared at least once more across 10 iterations
• 39% never reappeared at all

This bimodal pattern is significant: hallucinations are either highly stable and predictable or entirely unpredictable, with little middle ground.

Why Hallucinations Repeat:

1. Deterministic patterns: Given similar context, models produce similar outputs. "How do I connect to a PostgreSQL database in Python?" tends to generate similar import suggestions.

2. Common prompts: Developers ask similar questions. "Add logging to Flask" or "Parse JSON in Node" are near-universal tasks with common phrasings.

3. Naming convention convergence: Multiple models trained on similar data arrive at similar "plausible" names for packages that don't exist.

4. Temperature effects: At lower temperature settings (more deterministic), models are more likely to produce identical outputs across invocations.

Practical Implication:

An attacker can:

1. Query AI models with common development questions
2. Collect hallucinated package names from responses
3. Filter for names that appear repeatedly
4. Register those names with high confidence they'll be requested by real developers

This transforms a random-seeming vulnerability into a systematic attack.

Real-World Proof: The huggingface-cli Experiment¶

The theoretical attack path described above has been validated in practice. In March 2024, security researcher Bar Lanyado of Lasso Security³ discovered that multiple LLMs consistently hallucinated a Python package called huggingface-cli. While Hugging Face is a major AI platform with a legitimate CLI tool (installed via pip install -U "huggingface_hub[cli]"), no package named huggingface-cli existed on PyPI.

Lanyado registered the name on PyPI with an empty, benign package—no code, no README, no search optimization. Within three months, it was downloaded over 30,000 times. The downloads were not from automated scanners; they came from developers following AI-generated installation instructions.

The impact extended beyond individual developers. Alibaba's GraphTranslator project included pip install huggingface-cli in its official installation documentation—instructions that originated from AI-generated content. Even a Hugging Face-owned project briefly referenced the fake package before being notified.

This experiment demonstrated every phase of the slopsquatting attack lifecycle with a benign payload. Had the package contained malicious code, thousands of developer environments—including those at major technology companies—would have been compromised.

The Attack Lifecycle¶

Slopsquatting attacks follow a predictable lifecycle:

Phase 1: Reconnaissance

Attackers systematically query AI coding assistants:

• Common development tasks across languages
• Popular frameworks and their typical integrations
• Security and authentication patterns
• Database and API integration scenarios

They collect every package name mentioned, checking each against actual registries.

Phase 2: Name Harvesting

Attackers filter collected names:

• Remove packages that actually exist
• Track which non-existent names appear repeatedly
• Prioritize by estimated frequency and value of target contexts
• Consider cross-model consistency (names hallucinated by multiple AI tools)

Phase 3: Registration

Attackers register high-value hallucinated names:

• Create packages on PyPI, npm, and other registries
• Include legitimate-seeming metadata and descriptions
• Add just enough functionality to avoid immediate suspicion
• Optionally include malicious payloads or credential harvesting

Phase 4: Passive Waiting

Unlike active attacks requiring victim targeting, slopsquatting is passive:

• Attackers wait for developers to receive AI suggestions
• Developers install packages without verification
• Malicious code executes on developer machines or in CI/CD
• Credentials, source code, or access are exfiltrated

Phase 5: Exploitation

Harvested credentials enable further attacks:

• Access to private repositories
• Publishing rights to legitimate packages
• Cloud infrastructure access
• Lateral movement through development systems

Comparison to Traditional Typosquatting¶

Slopsquatting and typosquatting exploit different error sources but share operational similarities:

Critical Distinction:

Typosquatting detection relies on proximity to known packages—lodahs is close to lodash. Slopsquatting involves names that may have no relationship to existing packages. A hallucinated flask-auth-middleware isn't a typo of anything—it's a plausible name that happens not to exist.

This distinction defeats many typosquatting defenses. Edit distance checking, keyboard proximity analysis, and visual similarity detection don't help when the attack targets novel names rather than variations of existing ones.

Defense Strategies¶

Defending against slopsquatting requires interventions at multiple points:

Real-Time Validation:

The most effective defense is validating packages before installation:

# Before: Install whatever AI suggested
pip install flask-auth-middleware

# After: Verify package exists and matches expectations
pip index versions flask-auth-middleware  # Check if it exists
pip install flask-auth-middleware==1.2.3  # Pin specific verified version

Tools can automate this:

• Pre-installation hooks: Scripts that verify package existence and metadata before allowing installation
• IDE plugins: Real-time checking of import statements against registries
• CI/CD gates: Pipeline steps that fail if unverified packages are introduced

Curated Package Lists:

Organizations can maintain approved package lists:

• Allowlists: Only packages on the approved list can be installed
• Internal registries: Proxy registries that only mirror vetted packages
• Package governance: Formal process for adding new dependencies

This is operationally expensive but effective for high-security environments.

AI Tool Integration:

AI assistants themselves could help:

• Real-time registry checking: Validate suggested packages against live registries before presenting to users
• Confidence indicators: Flag suggestions the model is uncertain about
• Warning annotations: Explicitly note when suggested packages couldn't be verified

Research suggests these approaches are viable. Lanyado's work found that some models can self-detect their own hallucinations: GPT-4 Turbo and DeepSeek achieved over 75% accuracy in identifying hallucinated package names they had just generated³. The Spracklen study tested more systematic mitigations—Retrieval Augmented Generation (RAG), self-detected feedback, and supervised fine-tuning—and reduced hallucination rates below 3% for some models⁴. These results suggest that provider-side mitigations are feasible, though widespread implementation remains limited.

Developer Training:

Awareness is a critical layer:

• Train developers to verify AI-suggested packages
• Emphasize that AI suggestions are not endorsements
• Build verification habits into development culture

Monitoring:

Detection after the fact:

• Monitor for installation failures (package doesn't exist)
• Alert on newly-created packages with suspicious characteristics
• Track dependencies against known-hallucinated name lists

Implementation Challenges¶

Several factors complicate defense:

Verification Friction:

Every verification step slows development. The appeal of AI assistants is speed—adding verification undermines that value proposition. Organizations must balance security with productivity.

Dynamic Package Ecosystem:

• New legitimate packages are created constantly
• A package that doesn't exist today might exist tomorrow
• False positives (blocking legitimate new packages) frustrate developers

Tooling Gaps:

Current development tools weren't designed for this threat:

• IDEs don't validate imports in real-time against registries
• Package managers don't distinguish "package doesn't exist" from "network error"
• CI/CD systems don't have standard slopsquatting detection

AI Tool Opacity:

Developers often can't configure how AI assistants generate suggestions. The model's behavior is largely fixed by the provider.

The Convenience-Security Tension¶

Slopsquatting exemplifies a broader tension in AI-assisted development: the features that make AI useful also make it dangerous.

Convenience Drivers:

• AI suggestions reduce typing and research time
• Developers accept suggestions without deep investigation
• Productivity metrics reward speed over verification

Security Requirements:

• Every dependency is a trust decision
• Verification requires interrupting flow
• Caution conflicts with AI tool value proposition

Possible Resolutions:

1. Invisible verification: Tools that validate without user intervention
2. Progressive trust: Reduced verification for well-established packages, full verification for unknowns
3. Shared intelligence: Community databases of known hallucinations
4. Provider responsibility: AI tool providers taking responsibility for suggestion validity

Agentic Dependency Injection: When AI Installs Without Asking¶

The threat landscape shifts dramatically when AI moves from suggesting packages to installing them. AI coding agents—autonomous systems that can execute commands, modify files, and run build processes—amplify slopsquatting risk by removing the human verification step entirely.

The Agentic Difference:

How agentic injection works:

1. User requests: "Set up a Flask app with authentication"
2. Agent generates code including from flask_auth_helper import AuthManager
3. Agent encounters ImportError when running code
4. Agent "fixes" the error by running pip install flask-auth-helper
5. Malicious package (registered by attacker) executes on installation
6. Agent continues, unaware that compromise occurred

The feedback loop vulnerability:

AI agents are designed to iterate until tasks complete. When an import fails:

• Agent assumes it needs to install a dependency
• Agent has credentials/access to run pip install or npm install
• Installation succeeds (malicious package exists)
• Agent proceeds, treating the task as successful
• Malicious code has executed with agent's privileges

This creates a self-reinforcing attack path: the hallucination creates an error, the error triggers installation, installation executes malicious code.

Scale implications:

• Assistants: Attacker waits for individual developers to accept suggestions
• Agents: Attacker's package is installed automatically across all agent sessions that hallucinate that name

With Trend Micro research⁵ documenting how AI agents are increasingly deployed in automated development pipelines, the scale of potential impact grows exponentially.

Compounding factors:

1. Agent privileges: Agents often run with developer credentials, npm tokens, or cloud access
2. Autonomous operation: No human reviews each command the agent executes
3. Retry behavior: Agents will repeatedly try to "fix" import errors
4. Memory persistence: Some agents learn from sessions, potentially spreading hallucinations
5. CI/CD integration: Agents in pipelines can affect production builds

The "vibe coding" amplification:

When developers embrace "vibe coding"—rapidly accepting AI-generated code without deep review—they're implicitly trusting not just the code but every dependency the AI suggests. When agents execute autonomously, even this minimal review disappears.

Agentic-specific defenses:

# Example: Agent configuration with dependency controls
agent_config:
  permissions:
    package_install:
      mode: "allowlist"  # Only pre-approved packages
      allowlist_source: "approved-packages.txt"

  behaviors:
    on_import_error: "halt_and_report"  # Don't auto-install
    on_new_dependency: "require_approval"

  sandbox:
    network: "internal_only"
    filesystem: "workspace_only"

Critical controls for agentic development:

1. Disable auto-installation: Agents should not have permission to install packages without approval
2. Pre-approved package lists: Agents can only use packages from vetted allowlists
3. Sandbox execution: Agent package operations occur in isolated environments
4. Audit logging: Every package installation attempt is logged for review
5. Human checkpoints: Package additions require human approval before continuing

The emerging threat:

Industry analysis indicates that slopsquatting attacks are increasingly targeting agentic workflows specifically. Attackers recognize that:

• Agents remove human verification
• Agents have persistent credentials
• Agents can be systematically probed for hallucination patterns
• Compromised agent sessions provide rich access

As organizations deploy AI agents for development tasks, slopsquatting transitions from a nuisance to a critical vulnerability requiring architectural controls.

Liability and Responsibility: When AI Introduces Vulnerabilities¶

The rise of AI-assisted and AI-autonomous development raises fundamental questions about legal and organizational responsibility when AI-generated code introduces security vulnerabilities. Traditional software liability frameworks assume human decision-making at key points; AI disrupts these assumptions in ways that remain legally and practically unresolved.

The Responsibility Gap:

When a developer manually writes code that introduces a vulnerability, the chain of responsibility is relatively clear: the developer made a decision, the organization employs the developer, and established frameworks (employment law, professional standards, contractual obligations) assign accountability. AI-assisted development fractures this chain:

Current Legal Landscape:

Most AI coding tools operate under terms of service that disclaim liability for generated code:

• No warranty: AI providers typically disclaim any warranty that generated code is fit for purpose, secure, or free from defects
• User responsibility: Terms generally place responsibility for reviewing and validating AI output on the user
• Indemnification: Users often indemnify providers against claims arising from use of generated code

These disclaimers may face legal challenges as AI becomes more autonomous and the "user reviewed the output" assumption becomes less realistic.

Emerging Legal Theories:

Several legal frameworks may evolve to address AI-introduced vulnerabilities:

1. Product liability: If AI-generated code is treated as a "product," providers might face strict liability for defects. The EU's revised Product Liability Directive explicitly includes software, though application to AI-generated code remains untested.

2. Professional negligence: Organizations deploying AI for development might face negligence claims if they fail to implement reasonable safeguards (verification, sandboxing, human review).

3. Vicarious liability: Organizations might be held responsible for AI "actions" as they would be for employee actions—treating AI agents as analogous to employees or contractors.

4. Contractual allocation: Software contracts may increasingly include explicit provisions allocating liability for AI-generated components, with downstream customers requiring warranties about AI usage.

Organizational Risk Management:

Regardless of how legal liability ultimately develops, organizations should proactively manage AI-related security risks:

1. Document AI usage: Maintain records of which code was AI-generated or AI-assisted. This documentation supports incident investigation and may be relevant to liability determinations.

2. Implement verification requirements: Establish policies requiring human review of AI-generated code, particularly for security-sensitive components. Document that reviews occurred.

3. Contractual clarity: Address AI-generated code explicitly in software contracts—both as a vendor (disclosing AI usage) and as a customer (requiring disclosure and establishing liability allocation).

4. Insurance considerations: Consult with insurers about coverage for AI-related security incidents. Cyber insurance policies may need endorsements addressing AI-specific risks.

5. Establish AI governance: Create organizational policies governing AI tool usage, including approved tools, required safeguards, and prohibited use cases.

The Insurance Question:

Cyber insurance policies traditionally cover costs arising from security incidents, but AI-generated vulnerabilities create coverage questions:

• Does a policy cover incidents caused by AI-generated code?
• If AI-generated code is considered a "known defect" (given documented hallucination rates), does that affect coverage?
• Do policies require disclosure of AI tool usage?

Organizations should review policies carefully and discuss AI usage explicitly with insurers.

Future Trajectory:

The liability landscape for AI-generated code will likely clarify through some combination of:

• Litigation: Court decisions establishing precedents for AI-related liability
• Legislation: Laws specifically addressing AI liability (the EU AI Act may provide early examples)
• Industry standards: Emergence of "reasonable practices" for AI-assisted development that define the standard of care
• Contractual evolution: Market-driven allocation of AI-related risks through contract terms

For now, organizations operate in legal uncertainty. The prudent approach is to treat AI-generated code with heightened scrutiny, document AI usage and review processes, and structure contracts to address AI-related risks explicitly. The question is not whether liability frameworks will adapt to AI-assisted development, but whether your organization will be prepared when they do.

Recommendations¶

For Developers:

1. Never install AI-suggested packages blindly. Verify existence before installation by searching the official registry.

2. Check package metadata. Legitimate packages have histories—downloads, GitHub stars, maintainer information. Be suspicious of empty or very new packages.

3. Use lockfiles religiously. Pin exact versions of verified packages to prevent substitution.

4. Build verification habits. Make package verification as automatic as code review.

For Organizations:

1. Implement pre-installation checks. Add verification steps to development environments and CI/CD pipelines.

2. Consider package allowlisting. For high-security environments, maintain curated lists of approved packages.

3. Monitor for slopsquatting indicators. Alert on installation failures, newly-introduced unusual packages, or patterns matching known hallucinations.

4. Train developers. Ensure teams understand that AI suggestions require verification.

5. Evaluate AI tools on security features. Prefer tools that validate suggestions or provide confidence indicators.

For AI Tool Providers:

1. Validate suggestions against registries. Before presenting package suggestions, verify they exist.

2. Provide confidence signals. Indicate when package names might be uncertain.

3. Enable grounding. Allow users to configure authoritative package lists.

4. Share research. Publish findings on hallucination patterns to help the community defend.

For Registry Operators:

1. Flag newly-registered packages matching AI hallucination patterns. Coordinate with researchers tracking hallucinated names.

2. Consider publication friction for suspicious names. Require additional verification for packages with characteristics of slopsquatting.

3. Provide verification APIs. Make it easy for tools to validate package existence and legitimacy.

Slopsquatting represents a supply chain attack vector that exists because of AI adoption. As AI-assisted development becomes standard practice, defenders must adapt—building verification into workflows, developing detection capabilities, and accepting that AI suggestions are not endorsements. The 19.7% hallucination rate and bimodal repeatability pattern (43% fully repeatable, 39% never repeated) documented by researchers make clear that this is not a theoretical concern but an active threat requiring systematic response.