33.3: The Future of AI in Software Development¶

GitHub research has reported substantial productivity gains from Copilot in controlled tasks,¹ and industry surveys suggest AI-assisted code generation is becoming common in many organizations. Adoption has been rapid compared with many previous developer tools. The trajectory suggests a plausible future where a large share of code is AI-generated or AI-assisted, fundamentally transforming how software is created and, consequently, how software supply chains must be secured.

Beyond coding assistants, agentic AI systems—AI that operates autonomously to accomplish goals—are emerging as development tools. Systems like Claude Code, Devin, and similar platforms can write, test, and debug code with minimal human direction. These tools promise dramatic productivity gains but introduce novel security considerations. When AI autonomously makes coding decisions, introduces dependencies, and commits changes, traditional security review processes must evolve. The question is not whether AI will transform software development, but how organizations can harness this transformation while managing its security implications.

AI Coding Assistant Adoption Trajectory¶

AI coding assistants have achieved remarkable adoption rates, setting the stage for deeper integration.

Current state:

The AI coding assistant landscape has matured remarkably quickly. GitHub Copilot² leads in adoption with millions of users and integration across major IDEs including Visual Studio Code, JetBrains products, and Visual Studio. Claude Code offers extensive code generation capabilities both as a standalone tool and integrated into development workflows. Amazon Q Developer⁴ (formerly CodeWhisperer) provides AWS-integrated development assistance, understanding cloud infrastructure patterns. Google's Gemini Code Assist⁵ delivers deep integration with the Google ecosystem. Beyond these major players, purpose-built AI-native development environments like Cursor,⁶ Cody, and others are emerging, designed from the ground up around AI assistance rather than retrofitting AI into traditional IDEs.

Developer surveys show adoption accelerating across organization sizes, from individual developers experimenting with personal projects to enterprise-wide deployments standardizing on AI assistance.

Adoption drivers:

Multiple factors are accelerating AI coding assistant adoption. GitHub's controlled study found developers completed one programming task 55% faster with Copilot, and other studies and surveys report varied productivity gains—a potential advantage that organizations cannot ignore in competitive markets. The automation of boilerplate code reduces the tedium that drives developer burnout, freeing engineers to focus on interesting problems. Junior developers report learning patterns faster when AI suggests idiomatic code, accelerating their growth curves. Documentation generation tools can automatically explain complex code sections, reducing the maintenance burden. Real-time feedback on potential bugs catches issues before they enter version control, improving code quality at the point of creation.

These tangible benefits drive adoption despite the limitations that remain.

Current limitations:

Despite rapid adoption, significant limitations temper enthusiasm for AI-generated code. AI systems generate code that appears plausible and may even pass initial testing but contains subtle logical errors. Security blind spots are particularly concerning—AI models trained on public codebases may suggest vulnerable patterns they've seen frequently in training data, perpetuating common security mistakes. Context limitations mean AI lacks understanding of the full codebase architecture, leading to suggestions that work locally but create problems system-wide. The hallucination phenomenon causes AI to confidently invent APIs, libraries, or patterns that don't exist, requiring developers to verify every suggestion. Finally, generated code may not match project conventions for style, naming, or architecture, creating inconsistency that makes codebases harder to maintain.

These limitations mean AI remains an assistant, not a replacement, requiring human oversight and verification.

Trajectory projections:

Based on current adoption curves and capability improvements, we can sketch a plausible evolution of AI in software development. By 2025-2026, AI coding assistants are becoming standard developer tooling in many organizations. The 2027-2028 timeframe may see AI generating much of the routine code in AI-forward teams while humans increasingly focus on architectural decisions and code review. By 2029-2030, AI could be deeply integrated throughout development workflows—from planning and specification through deployment and monitoring. Beyond 2030, development without AI assistance may become less common, though adoption will vary by domain, regulation, and risk tolerance.

This trajectory has profound security implications—the code securing our systems will increasingly be AI-generated.

Agentic Development: Autonomous Coding Systems¶

Agentic AI development represents the next phase: AI systems that autonomously accomplish development goals rather than responding to individual prompts.

Emerging systems:

Several agentic development systems have emerged:

Claude Code³:

• Terminal-based autonomous development environment
• Reads, writes, and executes code autonomously
• Plans multi-step implementation tasks
• Handles testing and debugging iterations

Devin⁷ (Cognition):

• Full development environment controlled by AI
• Plans and executes complex development tasks
• Browses documentation, runs tests, deploys code
• Positioned as "AI software engineer"

OpenAI Codex and GPT-based agents:

• Task completion through multi-step reasoning
• Tool use including code execution and file manipulation
• Integration with development workflows

AutoGPT and open-source agents:

• Community-developed autonomous systems
• Variable capability and reliability
• Experimentation platform for agentic approaches

Capability levels:

Agentic systems operate at increasing capability levels:

Current production use is primarily at Assisted and Delegated levels, with Supervised deployment in controlled contexts.

Development workflow integration:

Agentic AI is entering development workflows:

• Feature implementation: AI implements features from specifications
• Bug fixing: AI diagnoses and fixes reported issues
• Test generation: AI creates comprehensive test suites
• Refactoring: AI modernizes legacy code
• Documentation: AI generates and maintains documentation

Security implications:

Agentic development creates new security dynamics:

• Dependency decisions: AI choosing what libraries to use
• Pattern selection: AI selecting security-relevant implementation patterns
• Configuration choices: AI setting security-relevant parameters
• Review volume: Generated code volume may exceed human review capacity
• Trust boundaries: Determining what AI should be trusted to do

CoSAI frames this as an identity and delegation problem, not only an AI quality problem. A coding agent that can clone repositories, open pull requests, call build systems, or modify cloud configuration is a non-human supply chain participant with its own lifecycle, audit trail, and delegated authority. Risk depends on both capability—from simple Q&A to state-changing planners—and impact—from public documentation to production control planes. For higher-impact workflows, organizations should treat agents as first-class identities, separate agent permissions from on-behalf-of user permissions, and prefer short-lived, task-scoped, revocable rights over broad standing access.¹¹

Security Implications at Scale¶

When AI generates code at scale, security considerations scale proportionally—or potentially faster.

Volume challenges:

AI code generation creates volume challenges:

• Review bottlenecks: Human review cannot scale with generation volume
• Test coverage: Generated code requires comprehensive testing
• Vulnerability density: More code means more potential vulnerabilities
• Attack surface: Larger codebases have larger attack surfaces

Organizations must develop approaches that maintain security without creating bottlenecks that eliminate productivity gains.

Pattern propagation:

AI learns patterns from training data, propagating both good and bad practices:

• Common vulnerabilities: AI may reproduce vulnerable patterns it learned
• Outdated practices: Training data may include deprecated security approaches
• Insecure defaults: AI may choose insecure configurations that appear in training data
• Copied flaws: Similar vulnerable code may appear across AI-assisted projects

Research has found⁸ that AI coding assistants can suggest vulnerable code patterns, including SQL injection, path traversal, and insecure cryptographic practices.

Dependency introduction:

AI introduces dependencies with security implications:

• Unknown packages: AI may suggest unfamiliar or risky packages
• Version selection: AI may choose outdated or vulnerable versions
• Unnecessary dependencies: AI may add dependencies for simple tasks
• Conflict creation: AI-suggested dependencies may conflict with existing security choices

Organizations need controls on AI-introduced dependencies beyond individual code suggestions.

Supply chain implications:

AI code generation affects supply chains:

• Provenance complexity: Distinguishing human from AI code
• Attribution challenges: Who is responsible for AI-generated vulnerabilities?
• Audit difficulty: Reviewing AI decision-making in code generation
• Reproducibility questions: AI outputs may vary between generations

MCP and agent tool dependencies:

The Model Context Protocol (MCP) is making external tools a normal part of AI development workflows. MCP servers can expose files, Git operations, databases, cloud APIs, and other sensitive capabilities to agents. As discussed in Book 1, Section 10.4, this makes MCP servers a new dependency class with familiar supply chain concerns and AI-specific ones. Malicious or "rug-pull" servers can change behavior after approval; tool descriptions and retrieved resources can become prompt-injection channels; confused-deputy flaws can cause one principal's authority to be misused for another; and poorly scoped servers can leak tokens or credentials.¹²

The practical implication is not that organizations should avoid MCP or equivalent tool protocols. It is that agent tools need dependency governance: inventory, provenance review, update control, credential scoping, tool-definition integrity checks where available, and enforcement at the client, server, and final API boundary.

Security-by-Design Opportunities¶

AI development tools present opportunities to embed security into the development process itself.

Built-in security checks:

AI tools can incorporate security by default:

• Vulnerable pattern recognition: Flagging insecure code as it's generated
• Secure alternatives: Suggesting secure implementations automatically
• Dependency vetting: Checking security of suggested packages
• Configuration validation: Ensuring security-relevant settings are correct

GitHub Copilot Autofix⁹ and other tools are implementing security features, though coverage remains incomplete.

Secure code generation:

AI can be trained and tuned for secure generation:

• Security-focused fine-tuning: Training on secure code examples
• Vulnerability avoidance: Explicit training to avoid known vulnerable patterns
• Best practice encoding: Embedding security best practices in model behavior
• Context-aware security: Adapting security recommendations to context

Policy enforcement:

AI tools can enforce organizational policies:

• Approved dependency lists: Only suggesting vetted packages
• Configuration standards: Generating code meeting security standards
• Architecture patterns: Following approved security architectures
• Compliance requirements: Helping satisfy regulatory requirements

Integrated security workflow:

AI enables integrated security throughout development:

Traditional: Code → Build → Security Scan → Fix → Deploy
AI-Integrated: AI generates secure code → Continuous validation → Human review → Deploy

Security can shift from post-development remediation toward generation-time prevention.

Investment opportunity:

Organizations can influence AI tool security through:

• Vendor selection: Choosing tools with strong security features
• Configuration: Enabling and requiring security checks
• Feedback: Reporting security issues to improve tools
• Customization: Fine-tuning models for organizational standards

The Changing Role of Human Developers¶

AI transforms what human developers do, with implications for security.

Role evolution:

Human developer activities are shifting:

Declining human role in:

• Writing routine, boilerplate code
• Implementing well-understood patterns
• Basic testing and documentation
• Simple bug fixes

Increasing human role in:

• Architecture and design decisions
• Security review and oversight
• Complex problem-solving
• AI output validation
• Policy and standards development

Security review transformation:

Code review evolves from generation to curation:

• Volume management: Reviewing AI output rather than human code
• Pattern recognition: Identifying AI-specific error patterns
• Judgment application: Deciding when AI suggestions are appropriate
• Context provision: Ensuring AI has necessary security context

Effective security review of AI-generated code requires different skills than reviewing human-written code.

Human oversight also becomes more fragile as single agents become orchestrated agent systems. A reviewer may be "on the loop"—watching summaries or approving checkpoints—without seeing sub-agent conversations, tool calls, or parallel work that shaped the final change. CoSAI warns that this can create an illusion of control: visibility into a chat transcript is not the same as cryptographic provenance, auditable delegation, or enforceable separation of duties. For software supply chains, meaningful oversight should include task traces, tool-call logs, and policy gates, not merely human presence in the conversation.¹³

Expertise requirements:

Developer expertise requirements change:

• Less: Syntax memorization, boilerplate patterns
• More: Security architecture, threat modeling, AI oversight
• Different: Understanding AI capabilities and limitations

Security expertise becomes more valuable as AI handles routine implementation.

Risk: Skill atrophy:

AI assistance creates skill atrophy risks:

• Junior developers may not learn foundational security concepts
• Pattern recognition skills may degrade with reduced practice
• Understanding of "why" may diminish as AI handles "how"
• Security intuition requires practice AI may reduce

Organizations must maintain developer security skills even as AI handles implementation.

New Attack Surfaces from AI Development¶

AI-integrated development creates novel attack surfaces.

Prompt injection attacks:

Attackers can target AI through crafted inputs:

• Malicious comments: Code comments designed to manipulate AI
• Poisoned documentation: README or docs that influence AI suggestions
• Context manipulation: Crafted code that changes AI behavior
• Indirect injection: Attacking AI through data it processes

Research demonstrates¹⁰ prompt injection can cause AI to generate malicious code, exfiltrate data, or subvert security controls.

Training data attacks:

AI models can be compromised through training data:

• Poisoning: Introducing malicious patterns into training data
• Backdoors: Hidden triggers causing specific AI behaviors
• Bias introduction: Skewing AI toward vulnerable patterns

Public code repositories that train AI models become attack targets.

Supply chain attacks on AI tools:

AI development tools themselves become targets:

• Model compromise: Attacking the AI model serving suggestions
• Tool compromise: Attacking the IDE integration or API
• Update attacks: Malicious updates to AI tools
• Configuration attacks: Manipulating AI tool settings

Organizations must secure AI tools as critical development infrastructure.

Trust boundary exploitation:

Agentic AI creates trust boundary questions:

• AI with file system access can modify arbitrary code
• AI with network access can exfiltrate information
• AI with execution capability can run arbitrary code
• AI with credential access can authenticate as users

Determining appropriate AI permissions requires careful trust modeling.

AI-Native Defense Opportunities¶

AI also enables new defensive capabilities for supply chain security.

Continuous code analysis:

AI can analyze code continuously:

• Real-time vulnerability detection as code is written
• Context-aware security suggestions
• Semantic understanding of security implications
• Cross-file and cross-project analysis

Intelligent dependency management:

AI can improve dependency security:

• Automated assessment of dependency security posture
• Intelligent upgrade recommendations
• Impact analysis for dependency changes
• Alternative suggestions for risky dependencies

Anomaly detection:

AI excels at detecting anomalies:

• Unusual code patterns suggesting compromise
• Behavioral anomalies in builds and deployments
• Unexpected changes in established codebases
• Patterns indicating social engineering

Security automation:

AI enables security automation:

• Automated remediation of known vulnerability patterns
• Intelligent triage reducing human workload
• Predictive identification of security issues
• Adaptive security controls responding to threats

Human-AI collaboration:

Effective defense combines human and AI capabilities:

Preparation Recommendations¶

We recommend organizations prepare for AI-transformed development through:

Immediate actions:

1. Evaluate AI tool security features selecting tools with strong security capabilities
2. Enable security controls in AI coding assistants already deployed
3. Train developers on AI security implications and oversight
4. Establish AI dependency policies governing AI-suggested packages
5. Implement AI code review processes appropriate for AI-generated code
6. Inventory agent tool integrations including MCP servers, plugins, and agent-to-agent connectors

Near-term planning:

1. Develop AI security guidelines establishing organizational standards
2. Build AI oversight capabilities for reviewing AI-generated code at scale
3. Assess agentic tool readiness evaluating organizational preparation
4. Define agent identities and delegation policies separating agent rights from user rights
5. Preserve CI/CD and GitOps boundaries preventing agents from self-approving or directly mutating production
6. Maintain human expertise preventing skill atrophy

Strategic positioning:

1. Engage with AI tool vendors influencing security feature development
2. Participate in standards for AI development security
3. Build AI-native defenses leveraging AI for security, not just development
4. Plan workforce evolution adapting skills and roles for AI collaboration
5. Monitor threat evolution tracking AI-specific attack development

For security leaders:

1. Assess AI adoption understanding current AI use in development
2. Identify AI risks evaluating security implications of AI tools
3. Develop governance for AI in development
4. Build detection for AI-related security issues
5. Communicate trajectory helping leadership understand strategic implications

The integration of AI into software development is already underway; the open question is how far it goes and under what governance. Organizations that proactively address security implications—building AI-native defenses, establishing appropriate governance, and evolving human roles—will be better positioned to capture productivity benefits while managing risks. Those that ignore the transformation may face both security and competitive challenges as the industry evolves.