25.1 Evaluating Vendor Supply Chain Practices¶

The software you purchase includes software your vendors purchased—and software their vendors purchased. The SolarWinds attack demonstrated this vividly: organizations that had never heard of SolarWinds Orion discovered they were exposed because their vendors used it.¹ Your security posture is only as strong as the weakest link in your extended supply chain, making vendor supply chain evaluation essential for managing organizational risk.

Traditional vendor security assessments focus on data handling, access controls, and incident response. Supply chain security adds new dimensions: How do vendors build their software? What dependencies do they include? How do they verify the integrity of their releases? These questions require different evaluation approaches than conventional third-party risk management.

This section provides a framework for assessing vendor supply chain security practices—during procurement, contract negotiation, and ongoing relationships.

Due Diligence Questionnaires¶

Questionnaires remain the foundation of vendor security assessment, providing structured information collection at scale. Several frameworks address supply chain security with varying depth and focus.

Questionnaire framework comparison:

SIG Questionnaire:

The Shared Assessments SIG questionnaire is widely accepted and includes software development and third-party management sections. Key supply chain-relevant domains:

• Application Security (Domain H): Secure development practices
• Third Party Management (Domain T): Vendor's vendor management
• System Development (Domain L): SDLC practices

Note: The SIG questionnaire has been updated in 2023-2024 to include "Nth Party Management" and "Supply Chain Risk Management" domains specifically addressing modern supply chain concerns. Even with these additions, supplement with supply chain-specific questions for comprehensive coverage.

CAIQ (CSA):

The Cloud Security Alliance's CAIQ aligns with the Cloud Controls Matrix (CCM) and focuses on cloud service security. Supply chain coverage is indirect:

• DevOps security controls
• Change management
• Vulnerability management

Limitation: CAIQ assumes cloud/SaaS context; less applicable to software products.

Supply chain-specific questionnaire additions:

Standard questionnaires need supplementation. Key questions to add:

# Software Composition and Dependencies

1. Do you maintain a Software Bill of Materials (SBOM) for your products?
   - If yes, what format (`SPDX`, `CycloneDX`)?
   - Can you provide SBOMs to customers upon request?

2. How do you evaluate third-party and open source components before inclusion?
   - What criteria do you use (security, maintenance, license)?

3. How do you monitor dependencies for newly disclosed vulnerabilities?
   - What is your SLA for addressing critical vulnerabilities in dependencies?

## Build and Release Integrity

4. What SLSA level do your build processes achieve?
   - Do you generate and publish build provenance?

5. How do you ensure the integrity of your release artifacts?
   - Are releases signed? What signing mechanism?
   - How can customers verify signatures?

6. Who has access to publish releases? How is that access controlled?

## Supply Chain Incident Response

7. How would you detect a compromise of your build system?

8. Describe your process for notifying customers of supply chain incidents.

9. Have you experienced any supply chain security incidents in the past 3 years?
   - If yes, describe and explain remediation.

Evidence Collection¶

Questionnaire responses are claims; evidence validates them. For supply chain security, specific artifacts demonstrate practice maturity.

Evidence request templates:

SBOM request template:

Subject: SBOM Request for [Product Name]

As part of our vendor security evaluation, we request a Software 
Bill of Materials (SBOM) for [Product Name] version [X.Y.Z].

Requested format: `CycloneDX` 1.7+ or `SPDX` 2.3+ (JSON preferred)

Required information:
- All direct and transitive dependencies
- Version information for each component
- License information
- Supplier/author information where available

We will use this SBOM to:
- Evaluate supply chain risk during procurement
- Monitor for vulnerabilities during our use of the product
- Meet our compliance and risk management requirements

Please provide the SBOM within [timeframe] or indicate if you 
cannot provide one and why.

Evaluating evidence quality:

Assessing Vendor Maturity¶

Beyond checking boxes, assess overall supply chain security maturity. Frameworks like SLSA (Supply-chain Levels for Software Artifacts) and OpenSSF Scorecard provide structured evaluation criteria.

Maturity assessment criteria:

SLSA (Supply-chain Levels for Software Artifacts):

SLSA provides graduated levels of build integrity assurance:

Ask vendors: "What SLSA level do your build processes achieve? Can you provide provenance attestations?"

OpenSSF Scorecard:

For vendors whose products include open source (most do), OpenSSF Scorecard provides automated assessment of repository security practices:

For vendors with public repositories, you can run Scorecard yourself: scorecard --repo=github.com/vendor/product

Maturity levels for vendor evaluation:

Assessment Methodology Options¶

Questionnaires and evidence review are desk-based. For critical vendors, deeper assessment may be warranted.

On-site and remote assessments:

Remote assessment structure:

For vendors that warrant deeper assessment but don't justify on-site visits:

1. Pre-assessment (1 week before):
2. Send questionnaire and evidence requests
3. Review documentation

4. Prepare targeted questions

5. Assessment call (1-2 hours):

6. Introduction and scope confirmation
7. Walk through key processes (build, release, vulnerability management)
8. Technical demonstration if possible

9. Clarifying questions on questionnaire responses

10. Follow-up (1 week after):

11. Request additional evidence
12. Clarify discrepancies
13. Document findings

Assessment interview questions:

## Build Process
- Walk me through how a code change becomes a released product
- Who can approve changes? Who can trigger builds? Who can publish releases?
- What happens if someone attempts an unauthorized release?

## Dependency Management
- How do you decide whether to add a new dependency?
- Show me how you learn about vulnerabilities in your dependencies
- What's your process when a critical vulnerability is disclosed?

## Incident Response
- Describe your most recent security incident (sanitized)
- How would you know if your build system were compromised?
- How would you notify customers of a supply chain issue?

Continuous Monitoring vs. Point-in-Time Assessment¶

Initial assessment provides a snapshot; continuous monitoring provides ongoing visibility.

Continuous monitoring approaches:

Continuous monitoring program:

Triggering reassessment:

Re-evaluate vendors outside normal cycles when: - Security incident reported (theirs or in their supply chain) - Major product changes - Acquisition or ownership change - Significant organizational restructuring - Contract renewal - Elevated use of vendor product

Red Flags and Deal-Breakers¶

Some findings should give pause; others should stop procurement.

Red flag indicators:

Deal-breakers (consider stopping procurement):

• History of supply chain incidents with inadequate response
• Inability to articulate software development security practices
• No incident notification capability or commitment
• Refuses to provide any security evidence
• Use of known-compromised dependencies without remediation plan
• Legal or contractual resistance to security requirements

Contextualizing findings:

Not every gap is disqualifying. Consider: - Criticality of the vendor: Higher criticality demands higher standards - Nature of the gap: Process gaps may be addressable; cultural gaps are harder - Vendor's response: Defensiveness vs. acknowledgment and improvement plans - Available alternatives: Can you get this capability elsewhere? - Compensating controls: Can you mitigate the risk through your own controls?

Vendor Tiering by Criticality¶

Not all vendors require the same scrutiny. Tier vendors to focus effort appropriately.

Vendor tiering by criticality:

Tiering criteria:

Evaluate vendors across dimensions:

Tiering example:

Critical: 
- Core ERP system vendor
- Customer identity provider
- Primary cloud infrastructure

High:
- Development tools (IDE, CI/CD platforms)
- Security tooling vendors
- Customer communication platforms

Medium:
- HR systems
- Internal productivity tools
- Marketing technology

Low:
- Office supplies
- Facility management
- Non-integrated SaaS tools

Recommendations¶

We recommend the following approach to vendor supply chain evaluation:

1. Supplement standard questionnaires with supply chain-specific questions: SIG and CAIQ provide foundation but lack depth on modern supply chain concerns. Add questions about SBOMs, build integrity, and dependency management.

2. Request evidence, not just responses: Questionnaire answers are claims. Request SBOMs, provenance attestations, and audit reports that validate practices.

3. Use maturity frameworks for assessment: SLSA levels and OpenSSF Scorecard provide structured, industry-recognized criteria for evaluating supply chain security maturity.

4. Tier vendors by criticality: Allocate assessment effort proportionally. Critical vendors warrant deep assessment and continuous monitoring; low-risk vendors need abbreviated review.

5. Implement continuous monitoring for critical vendors: Point-in-time assessment decays immediately. Track vendor SBOMs, security ratings, and breach disclosures continuously.

6. Define red flags and deal-breakers before assessment: Know what findings are unacceptable before you start. This prevents pressure to overlook serious issues during procurement.

7. Document and communicate requirements early: Share your supply chain security expectations during RFP. This filters out unprepared vendors and gives serious vendors time to prepare evidence.

8. Treat assessment as relationship-building: The goal isn't to catch vendors failing—it's to establish mutual understanding of security expectations and build confidence in the relationship.

Vendor supply chain evaluation is an evolving discipline. As supply chain attacks increase, expectations will rise. Vendors who invest in supply chain security today will be better positioned for procurement conversations tomorrow—and organizations that evaluate rigorously will face lower risk from their extended supply chain.