Project 7: Rust Quality Lab

Build a layered quality harness using unit tests, integration tests, property testing, fuzzing, and benchmarks.

Quick Reference

Attribute Value
Difficulty Level 3: Advanced
Time Estimate 1-2 weeks
Main Programming Language Rust
Alternative Programming Languages Go, Python
Coolness Level Level 4: Hardcore Tech Flex
Business Potential 3. The “Service & Support” Model
Prerequisites Projects 1-6
Key Topics Unit tests, integration tests, proptest, cargo fuzz, criterion

1. Learning Objectives

  1. Build a risk-based quality matrix for a Rust crate.
  2. Define and test invariants with property testing.
  3. Harden parser boundaries with fuzz targets.
  4. Detect performance drift with statistical benchmarks.

2. Theoretical Foundation

2.1 Core Concepts

  • Unit tests validate local logic quickly.
  • Integration tests verify public API behavior.
  • Property tests validate invariants over generated inputs.
  • Fuzzing probes adversarial input surfaces.
  • Benchmarks enforce performance budgets.

2.2 Why This Matters

Many production failures are logic or edge-case errors, not type errors. Layered validation catches defects before customers do.

2.3 Common Misconceptions

  • “Rust safety makes fuzzing unnecessary” -> false.
  • “Property tests replace integration tests” -> false.
  • “Benchmarking is optimization theater” -> false when used for regression gating.

3. Project Specification

3.1 What You Will Build

A quality harness around a parser/transform crate with:

  • Unit and integration suites
  • proptest invariants
  • cargo fuzz target and corpus
  • Criterion benchmarks with baseline policy

3.2 Functional Requirements

  1. All declared invariants are mapped to tests.
  2. Fuzz target runs for a fixed budget with no crashes.
  3. Bench report compares current vs baseline.
  4. Failures generate actionable reproductions.

3.3 Non-Functional Requirements

  • Reliability: deterministic failure reproduction via seeds/corpus.
  • Speed: fast local checks, heavier checks in CI tiers.
  • Traceability: each found defect maps to test artifact.

3.4 Example Usage / Output

$ cargo test
running 128 tests
128 passed; 0 failed

$ cargo fuzz run parse_target -- -max_total_time=30
INFO: no crashes found in 30s, corpus=214 inputs

$ cargo bench
parser_small/throughput   time:   [1.21 us 1.24 us 1.28 us]
parser_large/throughput   time:   [9.87 ms 10.02 ms 10.18 ms]

3.5 Real World Outcome

You can prove your crate is correct under examples, generated inputs, mutated attacks, and performance budgets. Release confidence becomes evidence-based.

4. Solution Architecture

4.1 High-Level Design

Risk Map -> Test Matrix -> Execution Tiers
             │
             ├─ Unit + Integration (fast)
             ├─ Property (medium)
             ├─ Fuzzing (slow/adversarial)
             └─ Benchmarks (regression gate)

4.2 Key Components

Component Responsibility Key Decision
Invariant registry Source of quality truth One invariant = one or more tests
Fuzz target Untrusted input boundary Keep entrypoint minimal
Benchmark suite Perf drift detection Stable fixtures and thresholds
CI tiers Runtime budgeting Split fast and heavy jobs

5. Implementation Guide

5.1 The Core Question You’re Answering

“How do I prove correctness and robustness beyond hand-picked examples?”

5.2 Concepts You Must Understand First

  1. Deterministic test design.
  2. Generator constraints for property tests.
  3. Fuzz corpus lifecycle.
  4. Benchmark variance handling.

5.3 Questions to Guide Your Design

  1. Which invariants are critical enough to block release?
  2. Which API boundary is highest-risk for fuzzing?
  3. What constitutes a meaningful benchmark regression?

5.4 Thinking Exercise

Pick one parser invariant and write:

  • example test
  • property definition
  • fuzz hypothesis
  • benchmark metric impacted by violations

5.5 The Interview Questions They’ll Ask

  1. “How do you choose fuzz targets in Rust codebases?”
  2. “What is shrinking and why does it matter?”
  3. “How do you avoid flaky benchmarks?”
  4. “When should property tests fail CI immediately?”

5.6 Hints in Layers

  • Hint 1: Define invariants before writing test code.
  • Hint 2: Keep generators realistic, not fully random.
  • Hint 3: Persist crashing fuzz cases as permanent regressions.
  • Hint 4: Pin benchmark environment and fixtures.

5.7 Books That Will Help

Topic Book Chapter
Rust testing workflow “The Rust Programming Language” Ch. 11
Practical quality engineering “Effective Rust” Testing-related items
Benchmark discipline Criterion docs User guide

6. Testing Strategy

  • Unit: 60% of suite, deterministic and fast.
  • Integration: public behavior and CLI/API contracts.
  • Property: invariant coverage for parser and state transitions.
  • Fuzz: byte-stream adversarial mutation.
  • Benchmark: p50/p95 latency and throughput trends.

7. Common Pitfalls & Debugging

Pitfall Symptom Solution
Overbroad generators flaky/slow property tests constrain domains and add assumptions
Missing corpus management repeated rediscovery of crashes version corpus + replay in CI
Benchmark noise false regression alarms repeated runs + confidence thresholds

8. Self-Assessment Checklist

  • Every high-risk invariant has at least one automated check.
  • Fuzz targets and corpus are reproducible.
  • Benchmark thresholds are explicit and reviewable.
  • Quality report explains failures and fixes.

9. Completion Criteria

Minimum Viable Completion

  • Unit/integration/property suites pass.
  • Fuzz target runs baseline budget with no crash.

Full Completion

  • Benchmark baselines and regression policy documented.

Excellence

  • CI tiers optimize runtime while preserving confidence.