14.1 Fuzz Testing Dependencies Before Adoption¶

In 2014, security researcher Neel Mehta at Google discovered the Heartbleed vulnerability in OpenSSL through code review—a bug that had existed for two years in one of the internet's most critical libraries.¹ In the years since, automated fuzzing has become one of the most effective techniques for finding security vulnerabilities that human reviewers and traditional testing miss, with Google's OSS-Fuzz project alone finding over 10,000 vulnerabilities in critical open source projects.² When evaluating dependencies for your supply chain, checking whether a project undergoes continuous fuzzing—or running your own fuzz campaigns—provides evidence about code robustness that vulnerability databases cannot.

This section introduces fuzzing as a dependency evaluation technique, covering tools, integration approaches, and how to contribute findings back to the open source ecosystem.

Fuzzing Fundamentals¶

Fuzzing (or fuzz testing) is an automated testing technique that feeds random, malformed, or unexpected inputs to a program to discover bugs that cause crashes, hangs, or security vulnerabilities.

How Fuzzing Works:

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│  Fuzzer Engine  │────▶│  Target Code    │────▶│  Monitor        │
│  (generates     │     │  (library,      │     │  (detects       │
│   inputs)       │     │   function)     │     │   crashes)      │
└─────────────────┘     └─────────────────┘     └─────────────────┘
         │                                               │
         │◀──────────────── Feedback ───────────────────┘
         │                (coverage, crashes)

Modern fuzzers use coverage-guided fuzzing: they monitor which code paths inputs exercise and mutate inputs to explore new paths. This intelligent exploration finds edge cases that random input generation would miss.

What Fuzzing Finds:

Why Fuzzing Matters for Supply Chains:

Traditional security testing (code review, static analysis) misses bugs that only manifest with specific inputs. Fuzzing complements these techniques by:

• Exploring input spaces too large for manual testing
• Finding implementation bugs that reviewers overlook
• Providing evidence of code robustness
• Discovering vulnerabilities before attackers do

Security-focused organizations prioritize fuzzing parsing code in dependencies, where most memory safety bugs concentrate. Parsers handle untrusted input with complex state machines, making them prime targets. Extended fuzzing campaigns (months, not days) routinely discover crashes in popular libraries before they reach production.

Fuzzing Tools¶

Several fuzzing tools are available, each with different strengths.

OSS-Fuzz (Google):

OSS-Fuzz is Google's continuous fuzzing infrastructure for critical open source projects.

• Coverage: 1,000+ projects, 50,000+ bugs found, 13,000+ security vulnerabilities identified
• Cost: Free for accepted projects
• Infrastructure: Google-provided compute, continuous operation
• Integration: Automatic issue filing, tracking

Checking if a Dependency is in OSS-Fuzz:

# Check OSS-Fuzz project list via GitHub API
curl -s "https://api.github.com/repos/google/oss-fuzz/contents/projects" | \
  jq -r '.[].name' | grep -i "project-name"

# Or browse directly:
# https://github.com/google/oss-fuzz/tree/master/projects

# For a specific project, check if its directory exists:
curl -s "https://api.github.com/repos/google/oss-fuzz/contents/projects/openssl" | \
  jq -r '.name' 2>/dev/null && echo "Project is in OSS-Fuzz" || echo "Not found"

Projects in OSS-Fuzz receive ongoing security testing. This is a positive signal when evaluating dependencies.

AFL (American Fuzzy Lop):

AFL is the foundational coverage-guided fuzzer, now maintained as AFL++.

• Best for: C/C++ binaries with file input
• Approach: Genetic algorithm for input mutation
• Strengths: Battle-tested, extensive documentation

# Basic AFL++ usage
# Compile with instrumentation
afl-clang-fast -o target target.c

# Run fuzzer
afl-fuzz -i seeds/ -o output/ ./target @@

libFuzzer:

libFuzzer is LLVM's in-process fuzzer, integrated into the compilation toolchain.

• Best for: Library functions, APIs
• Approach: In-process, coverage-guided
• Strengths: Fast, low overhead, integrates with sanitizers

// Example fuzz target
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    // Call the function you want to fuzz
    parse_input(data, size);
    return 0;
}

# Compile with fuzzer and sanitizers
clang++ -fsanitize=fuzzer,address -o fuzz_target fuzz_target.cpp

# Run
./fuzz_target corpus/

Jazzer (Java):

Jazzer brings coverage-guided fuzzing to the Java ecosystem.

• Best for: Java and JVM languages
• Approach: Bytecode instrumentation
• Strengths: Works with existing Java code

// Example Jazzer fuzz target
class MyFuzzer {
    public static void fuzzerTestOneInput(byte[] data) {
        try {
            MyParser.parse(data);
        } catch (ParseException e) {
            // Expected - not a bug
        }
    }
}

Atheris (Python):

Atheris is Google's fuzzer for Python code.

• Best for: Python libraries
• Approach: Coverage-guided, native extension support

import atheris
import sys

def test_one_input(data):
    fdp = atheris.FuzzedDataProvider(data)
    my_library.parse(fdp.ConsumeString(100))

atheris.Setup(sys.argv, test_one_input)
atheris.Fuzz()

Tool Selection Guide:

Evaluating Dependency Robustness¶

Use fuzzing as part of dependency evaluation to assess code quality.

Pre-Adoption Fuzzing:

Before adopting a critical dependency:

1. Check OSS-Fuzz status: Is it already being fuzzed?
2. Review past findings: Check issue tracker for fuzzing-discovered bugs
3. Run quick fuzz campaign: Even a few hours can reveal issues

Quick Fuzz Campaign:

For a C library you're considering:

# 1. Clone and build with instrumentation
git clone https://github.com/project/library.git
cd library
CC=afl-clang-fast ./configure
make

# 2. Create minimal seed inputs
mkdir seeds
echo "minimal valid input" > seeds/seed1

# 3. Fuzz for 24 hours
timeout 86400 afl-fuzz -i seeds/ -o findings/ ./library-test @@

# 4. Analyze crashes
afl-collect findings/ crashes/

What Results Indicate:

Robustness Signals:

Beyond crash counts, look for:

• Sanitizer integration: Does the project test with ASan/MSan/UBSan?
• Fuzz targets in repo: Does fuzz/ directory exist?
• CI fuzzing: Is fuzzing part of continuous integration?
• Historical response: How quickly were past fuzzing bugs fixed?

Integrating Fuzzing into Vetting¶

Make fuzzing a formal part of your dependency vetting process.

Tiered Approach:

Vetting Workflow Integration:

# Dependency Vetting Checklist - Fuzzing

### For Critical Dependencies:

- [ ] Is the project in OSS-Fuzz?
  - Check: https://github.com/google/oss-fuzz/tree/master/projects

- [ ] If not in OSS-Fuzz, are there fuzz targets in the repo?
  - Check: `fuzz/`, `fuzzing/`, `tests/fuzz/` directories

- [ ] Review fuzzing bug history
  - Check issue tracker for "fuzzing", "fuzz", "crash"
  - Assess fix velocity and quality

- [ ] Run internal fuzz campaign (if critical and no evidence of fuzzing)
  - Duration: minimum 24 hours, ideally 1 week
  - Document findings and report upstream

- [ ] Decision: Accept / Reject / Require upstream improvements

Automated Checks:

#!/bin/bash
# check_fuzzing_status.sh

PROJECT=$1

# Check OSS-Fuzz
if curl -s "https://api.github.com/repos/google/oss-fuzz/contents/projects/$PROJECT" | grep -q "name"; then
    echo "✓ Project is in OSS-Fuzz"
else
    echo "✗ Not in OSS-Fuzz"
fi

# Check for fuzz targets in repo
if curl -s "https://api.github.com/repos/$PROJECT/contents" | grep -qiE "fuzz|fuzzing"; then
    echo "✓ Fuzz directory found"
else
    echo "? No fuzz directory detected"
fi

Contributing Fuzzing Results Upstream¶

When your fuzzing discovers bugs, contributing findings upstream improves the ecosystem.

Responsible Disclosure:

Fuzzing often finds security vulnerabilities. Follow responsible disclosure:

1. Report privately: Use security contact (SECURITY.md)
2. Provide details: Crashing input, stack trace, analysis
3. Allow fix time: Typically 90 days for security issues
4. Coordinate disclosure: Work with maintainer on timeline

Contribution Types:

Writing Effective Bug Reports:

## Fuzzing Bug Report

### Summary
Heap buffer overflow in parse_header() triggered by malformed input.

### Environment
- Library version: 2.3.4
- Compiler: clang 15.0 with ASan
- Fuzzer: libFuzzer, 48-hour campaign

### Reproduction
```bash
./library_test crash-input-abc123

Crashing Input¶

Attached: crash-abc123.bin (17 bytes)

Stack Trace¶

ERROR: AddressSanitizer: heap-buffer-overflow
READ of size 4 at 0x... thread T0
    #0 parse_header src/parser.c:147
    #1 process_input src/main.c:89

Analysis¶

The parser reads 4 bytes at offset specified in header without bounds checking. Malformed header with offset > buffer_size triggers out-of-bounds read.

Suggested Fix¶

Add bounds check before read at parser.c:145

**OSS-Fuzz Integration:**

For critical projects not in OSS-Fuzz, consider contributing integration:

```bash
# Create OSS-Fuzz project structure
oss-fuzz/
├── projects/
│   └── myproject/
│       ├── Dockerfile     # Build environment
│       ├── build.sh       # Build script
│       └── project.yaml   # Project metadata

Benefits of getting a project into OSS-Fuzz: - Continuous fuzzing with Google's infrastructure - Automatic bug filing - Regression testing - Coverage reporting

Continuous Fuzzing Infrastructure¶

For critical dependencies, establish ongoing fuzzing rather than one-time campaigns.

Continuous Fuzzing Setup:

Option 1: ClusterFuzz (Google Infrastructure):

• OSS-Fuzz uses ClusterFuzz internally
• For accepted projects, runs continuously
• Automatic regression detection

Option 2: Self-Hosted Continuous Fuzzing:

# Docker-based continuous fuzzing
version: '3'
services:
  fuzzer:
    build: ./fuzzing
    volumes:
      - ./corpus:/corpus
      - ./crashes:/crashes
    command: |
      while true; do
        timeout 3600 ./fuzz_target corpus/
        sync
      done
    deploy:
      replicas: 4

Option 3: CI Integration:

# GitHub Actions fuzzing
name: Fuzz
on:
  schedule:
    - cron: '0 0 * * *'  # Daily

jobs:
  fuzz:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build fuzzer
        run: |
          clang++ -fsanitize=fuzzer,address -o fuzz fuzz.cpp

      - name: Download corpus
        uses: actions/download-artifact@v4
        with:
          name: fuzz-corpus
          path: corpus/

      - name: Fuzz
        run: |
          mkdir -p new_corpus
          timeout 3600 ./fuzz corpus/ new_corpus/ || true

      - name: Upload corpus
        uses: actions/upload-artifact@v4
        with:
          name: fuzz-corpus
          path: new_corpus/

Corpus Management:

Fuzzing effectiveness depends on good seed corpus:

# Build corpus from tests, examples, real data
find tests/ -type f -name "*.input" -exec cp {} corpus/ \;

# Minimize corpus (remove redundant inputs)
afl-cmin -i raw_corpus/ -o minimized_corpus/ ./target @@

# Merge new findings
./fuzz -merge=1 corpus/ new_findings/

Fuzzing Metrics and Success Indicators¶

Track metrics to assess fuzzing effectiveness.

Key Metrics:

Coverage Reporting:

# Generate coverage report
LLVM_PROFILE_FILE="fuzz.profraw" ./fuzz corpus/
llvm-profdata merge -sparse fuzz.profraw -o fuzz.profdata
llvm-cov show ./fuzz -instr-profile=fuzz.profdata -format=html > coverage.html

When to Stop Fuzzing:

Diminishing returns are expected—initial hours find most bugs.

Recommendations¶

For Security Practitioners:

1. Check OSS-Fuzz first. Before evaluating a critical dependency, verify if it's continuously fuzzed. OSS-Fuzz participation is a strong quality signal.

2. Run quick campaigns. Even 24 hours of fuzzing on entry points can reveal concerning issues.

3. Focus on parsers. Functions that parse untrusted input (JSON, XML, network protocols) are highest value targets.

For Developers:

1. Include fuzzing in vetting. Add fuzzing status to your dependency evaluation checklist.

2. Contribute findings upstream. When you find bugs, report them responsibly. Better yet, contribute fixes and fuzz targets.

3. Write fuzz targets for your code. Make your own libraries easier to fuzz. This helps your users and improves your security.

For Organizations:

1. Establish fuzzing infrastructure. For critical dependencies, continuous fuzzing provides ongoing assurance.

2. Define fuzzing requirements. Require OSS-Fuzz participation or internal fuzzing for security-sensitive dependencies.

3. Support OSS-Fuzz contributions. Help get your critical dependencies into OSS-Fuzz. The ecosystem benefits, and so do you.

Fuzzing provides evidence of robustness that other evaluation methods cannot. A library that has survived millions of malformed inputs is more trustworthy than one that hasn't been tested. While fuzzing doesn't guarantee security, its absence for critical dependencies is a warning sign worth heeding.