25.1 Evaluating Vendor Supply Chain Practices¶

The software you purchase includes software your vendors purchased—and software their vendors purchased. The SolarWinds attack demonstrated this vividly: organizations that had never heard of SolarWinds Orion discovered they were exposed because their vendors used it.¹ Your security posture is only as strong as the weakest link in your extended supply chain, making vendor supply chain evaluation essential for managing organizational risk.

Traditional vendor security assessments focus on data handling, access controls, and incident response. Supply chain security adds new dimensions: How do vendors build their software? What dependencies do they include? How do they verify the integrity of their releases? These questions require different evaluation approaches than conventional third-party risk management.

This section provides a framework for assessing vendor supply chain security practices—during procurement, contract negotiation, and ongoing relationships.

Due Diligence Questionnaires¶

Questionnaires remain the foundation of vendor security assessment, providing structured information collection at scale. Several frameworks address supply chain security with varying depth and focus.

Questionnaire Limitations and Failure Modes¶

Before examining specific questionnaire frameworks, organizations should understand how questionnaire-based assessment fails in practice. Questionnaires are necessary but insufficient—without awareness of their limitations, they can provide false assurance that increases rather than reduces risk.

Common failure modes:

Compliance theater: Vendors provide desired answers regardless of actual practices. When questionnaires ask "Do you have a vulnerability management process?" the answer is almost always "Yes"—even when the process consists of ignoring vulnerability reports until customers complain. Questionnaire responses reflect what vendors know you want to hear, not necessarily what they do.

Point-in-time decay: Questionnaire responses are snapshots. A vendor might have had good practices when they completed the questionnaire but has since experienced turnover, reorganization, or budget cuts. The questionnaire sitting in your vendor management system may describe an organization that no longer exists.

Checkbox mentality: Organizations treat completed questionnaires as "done"—vendor assessed, box checked, move on. The questionnaire becomes an end rather than a means to understanding actual risk. When a breach occurs, the organization discovers their questionnaire-approved vendor had practices that wouldn't survive scrutiny.

Asymmetric information: Vendors complete dozens or hundreds of questionnaires; your security team reviews their responses once. Vendors know which answers trigger follow-up questions and which close the conversation. Over time, vendor responses optimize for passing assessments rather than accurately describing practices.

Substituting process for judgment: The questionnaire framework provides structure, but someone must evaluate whether responses actually indicate security. "We use static analysis" could mean running a tool once that found 10,000 issues still unresolved, or running comprehensive analysis with issues remediated. The questionnaire can't distinguish.

Mitigating questionnaire limitations:

The goal isn't to abandon questionnaires—they remain valuable for structured information collection. The goal is to use them as one input among several, not as the entirety of vendor assessment. For critical vendors, questionnaires should initiate deeper investigation, not conclude it.

Questionnaire framework comparison:

SIG Questionnaire:

The Shared Assessments SIG questionnaire is widely accepted and includes software development and third-party management sections. Key supply chain-relevant domains:

• Application Security (Domain H): Secure development practices
• Third Party Management (Domain T): Vendor's vendor management
• System Development (Domain L): SDLC practices

Note: The SIG questionnaire has been updated in 2023-2024 to include "Nth Party Management" and "Supply Chain Risk Management" domains specifically addressing modern supply chain concerns. Even with these additions, supplement with supply chain-specific questions for comprehensive coverage.

CAIQ (CSA):

The Cloud Security Alliance's CAIQ aligns with the Cloud Controls Matrix (CCM) and focuses on cloud service security. Supply chain coverage is indirect:

• DevOps security controls
• Change management
• Vulnerability management

Limitation: CAIQ assumes cloud/SaaS context; less applicable to software products.

Supply chain-specific questionnaire additions:

Standard questionnaires need supplementation. Key questions to add:

# Software Composition and Dependencies

1. Do you maintain a Software Bill of Materials (SBOM) for your products?
   - If yes, what format (`SPDX`, `CycloneDX`)?
   - Can you provide SBOMs to customers upon request?

2. How do you evaluate third-party and open source components before inclusion?
   - What criteria do you use (security, maintenance, license)?

3. How do you monitor dependencies for newly disclosed vulnerabilities?
   - What is your SLA for addressing critical vulnerabilities in dependencies?

## Build and Release Integrity

4. What SLSA level do your build processes achieve?
   - Do you generate and publish build provenance?

5. How do you ensure the integrity of your release artifacts?
   - Are releases signed? What signing mechanism?
   - How can customers verify signatures?

6. Who has access to publish releases? How is that access controlled?

## Supply Chain Incident Response

7. How would you detect a compromise of your build system?

8. Describe your process for notifying customers of supply chain incidents.

9. Have you experienced any supply chain security incidents in the past 3 years?
   - If yes, describe and explain remediation.

Applying Shared Responsibility to AI Vendors¶

AI vendor assessment needs a shared-responsibility model because a single AI-enabled product may combine a foundation model, hosted inference service, agent framework, third-party datasets, and your own application logic. CoSAI's AI Shared Responsibility Framework extends cloud shared-responsibility concepts to AI systems and emphasizes assigning one accountable party per activity across the AI stack.² Use that model to replace the generic question "Is the AI system secure?" with more precise questions about which layer owns each control.

This boundary-setting improves procurement conversations. We recommend asking a model provider for model lineage, known limitations, evaluation results, and disclosure practices; asking a platform or framework provider for isolation, logging, tool-execution, and service-level evidence; and reserving your own controls for data access, use-case approval, monitoring, and incident response. A vendor can support your governance program, but it cannot decide which customer records your chatbot may retrieve or which AI-generated decisions your organization is willing to accept.

Evidence Collection¶

Questionnaire responses are claims; evidence validates them. For supply chain security, specific artifacts demonstrate practice maturity.

Evidence request templates:

SBOM request template:

Subject: SBOM Request for [Product Name]

As part of our vendor security evaluation, we request a Software 
Bill of Materials (SBOM) for [Product Name] version [X.Y.Z].

Requested format: `CycloneDX` 1.7+ or `SPDX` 2.3+ (JSON preferred)

Required information:

- All direct and transitive dependencies
- Version information for each component
- License information
- Supplier/author information where available

We will use this SBOM to:

- Evaluate supply chain risk during procurement
- Monitor for vulnerabilities during our use of the product
- Meet our compliance and risk management requirements

Please provide the SBOM within [timeframe] or indicate if you 
cannot provide one and why.

Evaluating evidence quality:

Assessing Vendor Maturity¶

Beyond checking boxes, assess overall supply chain security maturity. Frameworks like SLSA (Supply-chain Levels for Software Artifacts) and OpenSSF Scorecard provide structured evaluation criteria.

Maturity assessment criteria:

SLSA (Supply-chain Levels for Software Artifacts):

SLSA provides graduated levels of build integrity assurance:

Ask vendors: "What SLSA level do your build processes achieve? Can you provide provenance attestations?"

OpenSSF Scorecard:

For vendors whose products include open source (most do), OpenSSF Scorecard provides automated assessment of repository security practices:

For vendors with public repositories, you can run Scorecard yourself: scorecard --repo=github.com/vendor/product

Maturity levels for vendor evaluation:

Assessment Methodology Options¶

Questionnaires and evidence review are desk-based. For critical vendors, deeper assessment may be warranted.

On-site and remote assessments:

Remote assessment structure:

For vendors that warrant deeper assessment but don't justify on-site visits:

1. Pre-assessment (1 week before):
2. Send questionnaire and evidence requests
3. Review documentation

4. Prepare targeted questions

5. Assessment call (1-2 hours):

6. Introduction and scope confirmation
7. Walk through key processes (build, release, vulnerability management)
8. Technical demonstration if possible

9. Clarifying questions on questionnaire responses

10. Follow-up (1 week after):

11. Request additional evidence
12. Clarify discrepancies
13. Document findings

Assessment interview questions:

## Build Process

- Walk me through how a code change becomes a released product
- Who can approve changes? Who can trigger builds? Who can publish releases?
- What happens if someone attempts an unauthorized release?

## Dependency Management

- How do you decide whether to add a new dependency?
- Show me how you learn about vulnerabilities in your dependencies
- What's your process when a critical vulnerability is disclosed?

## Incident Response

- Describe your most recent security incident (sanitized)
- How would you know if your build system were compromised?
- How would you notify customers of a supply chain issue?

Continuous Monitoring vs. Point-in-Time Assessment¶

Initial assessment provides a snapshot; continuous monitoring provides ongoing visibility.

Continuous monitoring approaches:

Continuous monitoring program:

Triggering reassessment:

Re-evaluate vendors outside normal cycles when:

• Security incident reported (theirs or in their supply chain)
• Major product changes
• Acquisition or ownership change
• Significant organizational restructuring
• Contract renewal
• Elevated use of vendor product

Red Flags and Deal-Breakers¶

Some findings should give pause; others should stop procurement.

Red flag indicators:

Deal-breakers (consider stopping procurement):

• History of supply chain incidents with inadequate response
• Inability to articulate software development security practices
• No incident notification capability or commitment
• Refuses to provide any security evidence
• Use of known-compromised dependencies without remediation plan
• Legal or contractual resistance to security requirements

Contextualizing findings:

Not every gap is disqualifying. Consider:

• Criticality of the vendor: Higher criticality demands higher standards
• Nature of the gap: Process gaps may be addressable; cultural gaps are harder
• Vendor's response: Defensiveness vs. acknowledgment and improvement plans
• Available alternatives: Can you get this capability elsewhere?
• Compensating controls: Can you mitigate the risk through your own controls?

Vendor Tiering by Criticality¶

Not all vendors require the same scrutiny. Tier vendors to focus effort appropriately.

Vendor tiering by criticality:

Tiering criteria:

Evaluate vendors across dimensions:

Tiering example:

Critical: 

- Core ERP system vendor
- Customer identity provider
- Primary cloud infrastructure

High:

- Development tools (IDE, CI/CD platforms)
- Security tooling vendors
- Customer communication platforms

Medium:

- HR systems
- Internal productivity tools
- Marketing technology

Low:

- Office supplies
- Facility management
- Non-integrated SaaS tools

One category deserves scrutiny beyond what standard tiering suggests: security tooling vendors. Their products run inside CI/CD pipelines and on developer workstations with elevated access, and attackers have noticed. In 2026, attackers used credentials stolen in the March 19 supply chain attack on the open source Trivy scanner to compromise Checkmarx's GitHub repositories, then published malicious code through the vendor's VS Code extensions, GitHub Actions, Docker images, and a Jenkins plugin—channels that customers' pipelines consumed with the authority of a trusted security vendor. When evaluating security vendors, ask specifically how they secure their own artifact distribution channels, whether their extensions and actions are signed and pinnable, and how quickly they can notify customers of compromised releases.

Recommendations¶

We recommend the following approach to vendor supply chain evaluation:

1. Supplement standard questionnaires with supply chain-specific questions: SIG and CAIQ provide foundation but lack depth on modern supply chain concerns. Add questions about SBOMs, build integrity, and dependency management.

2. Request evidence, not just responses: Questionnaire answers are claims. Request SBOMs, provenance attestations, and audit reports that validate practices.

3. Use maturity frameworks for assessment: SLSA levels and OpenSSF Scorecard provide structured, industry-recognized criteria for evaluating supply chain security maturity.

4. Tier vendors by criticality: Allocate assessment effort proportionally. Critical vendors warrant deep assessment and continuous monitoring; low-risk vendors need abbreviated review.

5. Implement continuous monitoring for critical vendors: Point-in-time assessment decays immediately. Track vendor SBOMs, security ratings, and breach disclosures continuously.

6. Define red flags and deal-breakers before assessment: Know what findings are unacceptable before you start. This prevents pressure to overlook serious issues during procurement.

7. Document and communicate requirements early: Share your supply chain security expectations during RFP. This filters out unprepared vendors and gives serious vendors time to prepare evidence.

8. Treat assessment as relationship-building: The goal isn't to catch vendors failing—it's to establish mutual understanding of security expectations and build confidence in the relationship.

Vendor supply chain evaluation is an evolving discipline. As supply chain attacks increase, expectations will rise. Vendors who invest in supply chain security today will be better positioned for procurement conversations tomorrow—and organizations that evaluate rigorously will face lower risk from their extended supply chain.