Skip to content

33.2: Evolving Threats and Adversary Capabilities

The xz-utils backdoor attempt in March 2024 required years of patient trust-building by the attacker—creating legitimate contributions, building social capital, and waiting for the right moment to strike. Future attackers may not need such patience. AI tools can generate convincing code contributions at scale. Social engineering attacks can be personalized and automated. Nation-states are investing heavily in supply chain capabilities. The attack surface is expanding into IoT, edge computing, and embedded systems with weaker security foundations than traditional IT. Understanding how adversary capabilities are evolving helps organizations anticipate and prepare for threats beyond today's playbook.

The security field has long observed an offense-defense asymmetry—attackers need only find one weakness while defenders must protect everything. This asymmetry is shifting in complex ways. Some emerging technologies favor defenders; others empower attackers. AI assists both vulnerability discovery and exploitation. Automation enables both faster patching and faster attack development. The organizations that navigate this evolving landscape successfully will be those that understand not just today's threats but the trajectory of adversary capability development.

AI-Assisted Attack Development

Artificial intelligence is transforming attacker capabilities just as it transforms defense. AI tools lower barriers to sophisticated attacks, enable operations at unprecedented scale, and create novel attack vectors that didn't previously exist.

Code generation for attacks:

Large language models can assist attack development:

  • Vulnerability discovery: AI analyzing code to find exploitable flaws
  • Exploit development: Generating exploit code from vulnerability descriptions
  • Malware creation: Producing functional malicious code with natural language prompts
  • Obfuscation: Generating variants to evade detection
  • Automation scripts: Creating tools for attack automation

Research demonstrates that AI models can generate functional exploit code, though with limitations. As models improve, the gap between AI capability and human expert capability narrows.

Lowered barriers:

AI democratizes sophisticated attacks:

  • Script kiddies can generate competent attack code
  • Language barriers reduce as AI assists non-English speakers
  • Technical knowledge requirements decrease
  • Attack development time compresses
  • Iteration and refinement become faster

The implication is more attackers capable of more sophisticated operations—a quantitative and qualitative expansion of the threat landscape.

AI-enhanced vulnerability research:

Attackers will increasingly use AI for vulnerability discovery:

  • Automated analysis of code changes for security implications
  • Pattern matching across codebases to find similar vulnerabilities
  • Fuzzing guided by AI understanding of code semantics
  • Automated triage to prioritize most exploitable findings

Defenders use these same techniques, but attackers have advantages—they need only find vulnerabilities, not fix them, and they can focus on attack paths while defenders must address all issues.

Social Engineering at Scale

AI enables social engineering attacks at scales and sophistication levels previously impossible, with direct implications for supply chain security.

Personalized targeting:

AI can generate highly personalized social engineering:

  • Analysis of target's public communications, coding style, interests
  • Generated messages matching target's context and concerns
  • Convincing impersonation of trusted parties
  • Dynamic conversation handling responding naturally to replies

The xz-utils attack involved social engineering pressure on the maintainer through seemingly genuine community members. AI can generate such personas at scale—hundreds of "community members" pressuring maintainers, all AI-generated and individually convincing.

Trust exploitation:

Open source collaboration depends on trust that AI can exploit:

  • Contributor personas: AI-maintained identities building reputation over time
  • Discussion manipulation: AI generating supportive comments for malicious PRs
  • Community pressure: Coordinated AI personas pressuring maintainers
  • Documentation spoofing: AI-generated convincing but misleading documentation

Detection challenges:

AI-generated social engineering is difficult to detect:

  • No obvious markers distinguishing AI from human text
  • Sophisticated AI avoids patterns that flag automated content
  • Volume of open source communication makes monitoring difficult
  • Cultural norms favor assuming good faith

Countermeasures:

Defenses against AI social engineering include:

  • Identity verification for sensitive access
  • Multi-party approval for critical changes
  • Anomaly detection in contribution patterns
  • Community awareness training
  • Rate limiting and cooling periods for new contributors

However, determined attackers with AI assistance can likely bypass many countermeasures given sufficient investment.

Nation-State Investment

Nation-state investment in supply chain attack capabilities is increasing, driven by demonstrated effectiveness and strategic value.

Investment drivers:

Supply chain attacks appeal to nation-states because:

  • Scale: Single compromise reaches thousands of targets
  • Access: Reaches hardened networks through trusted software
  • Deniability: Attribution is difficult
  • Persistence: Embedded access can persist undetected
  • Cost efficiency: High return on investment compared to alternatives

SolarWinds demonstrated supply chain attacks can compromise the most sensitive targets—U.S. government agencies, major corporations, critical infrastructure operators. This success attracted increased investment.

Capability development:

Nation-state supply chain capabilities include:

  • Long-term infiltration: Multi-year operations building trust (xz-utils pattern)
  • Infrastructure compromise: Targeting build systems, registries, distribution
  • Zero-day stockpiles: Vulnerabilities held for strategic use
  • Insider recruitment: Developing sources within key projects
  • Technical expertise: Dedicated teams focused on supply chain vectors

Known actors:

Multiple nation-states have demonstrated supply chain capabilities:

  • Russia: SolarWinds (2020), NotPetya (2017)
  • North Korea (Lazarus Group): npm supply chain attacks targeting cryptocurrency developers including the "Contagious Interview" campaign (2024), with hundreds of malicious packages discovered
  • China (multiple groups including Silk Typhoon, TA415): Supply chain attacks targeting IT services, semiconductors, and managed service providers
  • Others: Capabilities likely exist beyond publicly attributed operations

Implications:

Increased nation-state investment means:

  • More sophisticated attacks
  • More patient, long-term operations
  • More resources for attack development
  • Critical infrastructure as primary targets
  • Geopolitical events triggering attack campaigns

Organizations in critical infrastructure, government, defense, and technology sectors should assume nation-state adversaries are actively targeting their supply chains.

Expanding Attack Surface

The attack surface for supply chain compromises is expanding beyond traditional software into IoT, edge computing, and embedded systems with fundamentally weaker security foundations.

IoT proliferation:

The Internet of Things creates massive attack surface:

  • Scale: Billions of devices, many with software supply chains
  • Diversity: Countless manufacturers, inconsistent practices
  • Longevity: Devices deployed for decades, rarely updated
  • Constraints: Limited resources for security controls
  • Visibility: Organizations often don't know what's on their networks

IoT supply chain compromises could affect physical systems—building controls, medical devices, industrial equipment, vehicles.

Edge computing:

Edge computing pushes processing to distributed locations:

  • Software deployed to thousands of edge locations
  • Update processes less controlled than central systems
  • Physical security often limited
  • Heterogeneous environments complicate management
  • Supply chains extend to edge-specific components

Edge compromises could affect operations at scale while evading central monitoring.

Embedded systems:

Embedded software in non-obvious devices creates hidden supply chains:

  • Firmware in enterprise hardware (servers, network equipment)
  • Embedded systems in vehicles
  • Industrial control systems
  • Medical devices
  • Building automation

These systems often use open source components but lack the security infrastructure of traditional IT systems.

Supply chain implications:

Attack surface expansion means:

  • More targets with weaker defenses
  • Longer vulnerability windows (infrequent updates)
  • Detection challenges (limited monitoring)
  • Physical consequences from software compromises
  • Extended attack dwell time in overlooked systems

Organizations must extend supply chain security thinking beyond traditional software to all software-containing systems in their environment.

Agentic AI Security Implications

Agentic AI—AI systems that take autonomous actions in the world—creates novel supply chain security challenges both as attack target and attack tool.

As attack target:

Agentic AI systems have unique vulnerabilities:

  • Prompt injection: Adversaries manipulating AI behavior through crafted inputs
  • Data poisoning: Compromising training data to influence AI decisions
  • Tool manipulation: Attacking tools that AI agents use
  • Context manipulation: Influencing AI through environment rather than direct attack
  • Goal subversion: Subtle modifications to AI objectives

Agentic AI in development workflows (AI coding assistants, autonomous testing, automated deployment) creates new supply chain attack vectors. Compromising the AI could affect all code it generates or reviews.

As attack tool:

Attackers can use agentic AI for supply chain attacks:

  • Autonomous reconnaissance: AI exploring attack surfaces continuously
  • Automated exploitation: AI developing and executing exploits
  • Adaptive attacks: AI adjusting tactics based on defender responses
  • Persistent operations: AI maintaining access without human direction
  • Scale operations: AI conducting attacks across many targets simultaneously

Current AI capabilities are limited, but trajectory suggests increasing autonomy in attack operations.

Supply chain for AI:

AI systems themselves have supply chains:

  • Training data sourced from various origins
  • Pre-trained models from external providers
  • Fine-tuning datasets potentially poisoned
  • AI frameworks and libraries with vulnerabilities
  • Deployment infrastructure for AI systems

Securing AI supply chains requires techniques beyond traditional software security.

Emerging concerns:

Novel AI attack patterns include:

  • Model stealing: Extracting proprietary models through APIs
  • Membership inference: Determining training data contents
  • Backdoor insertion: Hidden triggers causing specific AI behaviors
  • Adversarial examples: Inputs designed to cause AI errors

Organizations deploying AI must consider both using AI for security and securing AI itself.

Novel Attack Vectors

History suggests the most impactful future attacks will use vectors we haven't yet imagined. Prudent security planning considers speculative threats.

Speculative vectors:

Potential future attack vectors include:

Quantum-enabled attacks: - Cryptographic signatures broken, enabling forgery of software provenance - Historical traffic decryption revealing secrets - Random number generator compromise - Supply chain provenance systems undermined

Hardware supply chain: - Chips with hidden functionality - Compromised manufacturing equipment - Hardware trojans in commodity components - Firmware implants in devices

Emerging protocol compromise: - New standards with undiscovered flaws - Implementation inconsistencies enabling attacks - Transition periods creating vulnerability windows

Social/technical hybrid: - Long-term community infiltration at scale - Coordinated manipulation of multiple projects - Foundation or governance body compromise

Preparing for unknowns:

Organizations can prepare for unknown vectors through:

  • Defense in depth: Multiple layers surviving individual failures
  • Detection capability: Noticing anomalies even without specific signatures
  • Recovery capability: Bouncing back from successful attacks
  • Adaptability: Processes for responding to novel situations
  • Continuous learning: Incorporating new threat intelligence

Specific predictions will be wrong, but general resilience serves regardless of specific attack vectors.

The Offense-Defense Balance

The future balance between offensive and defensive capabilities is uncertain but critical for strategic planning.

Factors favoring offense:

  • AI lowering barriers to sophisticated attacks
  • Expanding attack surface through IoT/edge
  • Increasing value of supply chain targets
  • Nation-state investment in capabilities
  • Asymmetric economics (attackers need one success; defenders need total protection)

Factors favoring defense:

  • Memory-safe languages eliminating vulnerability classes (see CISA guidance)
  • Improved tooling for detection and response (SLSA, Sigstore)
  • Regulatory pressure driving security investment
  • Industry coordination on threats (OpenSSF)
  • Hardware security features providing stronger guarantees

Net assessment:

The near-term balance likely favors attackers:

  • Defensive technologies need time for adoption
  • Attack capabilities are advancing rapidly
  • Attack surface expansion outpaces defensive scaling
  • Coordination challenges slow defensive improvements

Longer term, the balance could shift:

  • Memory-safe languages eliminate major vulnerability categories
  • Formal verification provides stronger guarantees
  • Hardware security becomes ubiquitous
  • Industry coordination matures

Strategic implications:

Organizations should plan for:

  • Heightened threat environment in near term
  • Need for increased investment in defense
  • Emphasis on detection and response, not just prevention
  • Resilience as key security property
  • Continuous adaptation as landscape evolves

Preparation Recommendations

We recommend organizations prepare for evolving threats through:

Immediate actions:

  1. Assess nation-state exposure if in critical infrastructure, government, or technology sectors
  2. Extend security to IoT/edge including these systems in supply chain programs
  3. Train on AI-enabled social engineering raising awareness of new manipulation capabilities
  4. Implement defense in depth so no single compromise is catastrophic
  5. Build detection capability to notice anomalies beyond known signatures

Strategic planning:

  1. Assume sophisticated adversaries designing security for capable, patient attackers
  2. Invest in resilience accepting some attacks will succeed
  3. Monitor threat evolution maintaining awareness of emerging capabilities
  4. Plan for quantum transition beginning cryptographic agility efforts
  5. Extend attack surface visibility knowing all software-containing systems

For security leaders:

  1. Communicate evolving threat to leadership, adjusting risk assessments
  2. Advocate for investment matching threat trajectory
  3. Build adaptive capability not just static defenses
  4. Engage with community sharing and receiving threat intelligence
  5. Scenario plan for novel attack types

For policy makers:

  1. Support defensive technology development funding memory-safe transitions, verification research
  2. Enable information sharing reducing barriers to threat intelligence exchange
  3. Consider deterrence imposing costs on nation-state attackers
  4. Invest in workforce developing security expertise
  5. Coordinate internationally on shared threats

The threat landscape will continue evolving. Organizations that build adaptive security capabilities—continuously learning, adjusting, and improving—will navigate this evolution more successfully than those building static defenses against today's known threats.