Project 10: Capstone - Design Your Own Safe Abstraction

Design and build a Rust crate with a safe public API and carefully documented unsafe internals.

Quick Reference

Attribute Value
Difficulty Master
Time Estimate 1-3 months
Main Programming Language Rust
Alternative Programming Languages C++ (unsafe abstractions)
Coolness Level Extremely High
Business Potential High
Prerequisites All previous projects
Key Topics API design, unsafe invariants, testing, documentation

1. Learning Objectives

By completing this project, you will:

  1. Design a safe API that encodes invariants in types and lifetimes.
  2. Implement unsafe internals with documented proof obligations.
  3. Provide comprehensive tests, benchmarks, and examples.
  4. Publish a crate-quality README and documentation.

2. All Theory Needed (Per-Concept Breakdown)

2.1 Invariant-Driven API Design

Fundamentals

An invariant is a rule that must always hold for safety. A safe API encodes these invariants so they cannot be violated by users. This is the core of Rust’s safety story: unsafe code is permitted only when its invariants are upheld by a safe interface.

Deep Dive into the concept

Designing safe abstractions means thinking in proofs. You must define the invariants your unsafe code assumes, then design the public API so those invariants are always true. For example, an arena allocator assumes that memory never moves after allocation. Its API ties references to the arena’s lifetime, so users cannot use references after drop. The compiler enforces the lifetime relationship; your unsafe code only has to maintain the memory invariants.

A good approach is to start by writing the public API and its invariants in plain language. Each method should list what it guarantees and what it assumes. Then, use the type system to enforce as much as possible: lifetimes for scope, ownership for exclusivity, and newtypes for constrained values. Only after that should you write unsafe code. This is the inverse of how many people start (writing unsafe code first), but it leads to safer designs.

Another key principle is minimizing unsafe surface area. The more unsafe code you have, the harder it is to audit. Keep unsafe blocks small and local, and document them with the invariant they rely on. Use safe wrappers to prevent misuse. For example, instead of exposing raw pointers, expose typed handles or guard objects.

Finally, think about failure modes. If an invariant is violated, what happens? Ideally, it is impossible in safe code. But you should still add debug assertions and tests that attempt to break invariants. Tools like miri and loom can help detect UB or concurrency bugs.

How this fit on projects

This concept is the capstone itself and ties together lessons from Projects 1-9, especially on lifetime and ownership reasoning.

Definitions & key terms

  • Invariant: A condition that must always hold.
  • Unsafe boundary: The line between safe and unsafe code.
  • Proof obligation: The reasoning you must supply to justify unsafe code.

Mental model diagram (ASCII)

Public API (safe)
   | enforces invariants
Unsafe core (assumes invariants)

How it works (step-by-step)

  1. Define invariants in docs.
  2. Design types and lifetimes to enforce them.
  3. Implement unsafe core with minimal surface.
  4. Test invariants with adversarial inputs.

Minimal concrete example

// Safe API
pub fn alloc<'a>(&'a self, t: T) -> &'a T { ... }
// Unsafe core assumes buffer stability

Common misconceptions

  • “Unsafe code is always dangerous.” It is safe if invariants are upheld.
  • “Documentation is optional.” It is part of the safety contract.

Check-your-understanding questions

  1. What is the invariant of a connection pool guard?
  2. Why is minimizing unsafe blocks important?

Check-your-understanding answers

  1. A connection is owned by exactly one guard or the pool at any time.
  2. Smaller unsafe blocks are easier to audit and reason about.

Real-world applications

  • Allocators, collections, concurrency primitives.

Where you’ll apply it

References

  • Rustonomicon “Unsafe Code”.
  • Rust API Guidelines.

Key insights

A safe abstraction is a proof encoded in types and lifetimes.

Summary

Design the API first, then write the unsafe code that is justified by it.

Homework/Exercises to practice the concept

  1. Write invariants for a hypothetical ring buffer.
  2. Design an API that enforces them.

Solutions to the homework/exercises

  1. Invariants: fixed capacity, head/tail indices in range, no overwrite of unread data.
  2. Expose push/pop that check capacity and maintain indices.

2.2 Testing Unsafe Code

Fundamentals

Unsafe code requires stronger testing: unit tests, property-based tests, and tools like miri to detect undefined behavior. Deterministic tests are crucial because many failures are rare.

Deep Dive into the concept

Testing unsafe code is about validating invariants. For memory safety, you need tests that stress allocations and deallocations, ensure no use-after-free, and validate alignment. For concurrency, you need tests that explore different interleavings. loom is a tool that runs concurrent code in a deterministic model, exploring possible schedules. miri interprets Rust code and detects UB such as use-after-free or invalid pointer deref.

You should also use property-based testing (e.g., proptest) to generate random sequences of operations. For a data structure, properties might include: length never negative, elements returned are among those inserted, no duplicates in a set, etc. For memory safety, you can include tests that insert and remove many elements in random order and run under miri.

Testing also includes documentation tests. For a public crate, examples in docs should compile and run. This ensures your API is usable and correctly documented.

How this fit on projects

This concept is applied in §6 (testing strategy) and §11 (self-assessment). It connects to Project 8 (concurrency testing) and Project 7 (pinning).

Definitions & key terms

  • Miri: Interpreter for detecting UB.
  • Loom: Deterministic concurrency testing tool.
  • Property-based testing: Testing with randomized inputs.

Mental model diagram (ASCII)

Unsafe code -> invariants -> tests (unit + property + miri)

How it works (step-by-step)

  1. Define invariants.
  2. Write tests that try to break them.
  3. Run miri and loom.
  4. Add property-based tests for sequences.

Minimal concrete example

#[test] fn no_use_after_free() { ... }

Common misconceptions

  • “If it compiles, unsafe is correct.” It is not.
  • “Unit tests are enough.” Concurrency requires more.

Check-your-understanding questions

  1. Why is miri important?
  2. What does loom do?

Check-your-understanding answers

  1. It detects undefined behavior at runtime.
  2. It explores possible thread interleavings deterministically.

Real-world applications

  • Lock-free data structures.
  • Custom allocators.

Where you’ll apply it

References

  • Miri docs.
  • Loom docs.

Key insights

Testing unsafe code means testing invariants, not just outputs.

Summary

Use specialized tools and property tests to validate unsafe code.

Homework/Exercises to practice the concept

  1. Write a property test for a stack.
  2. Run it under miri.

Solutions to the homework/exercises

  1. Ensure pop returns items that were pushed.
  2. Use cargo miri test.

2.3 Documentation and Safety Contracts

Fundamentals

Unsafe code requires explicit documentation of safety contracts: what invariants the user must uphold and what the code guarantees. This documentation is part of the API’s correctness.

Deep Dive into the concept

In Rust, unsafe does not mean “dangerous to use”; it means “the compiler cannot verify this.” The burden is on you to document the conditions under which the unsafe code is safe. This is especially important in public APIs: if you expose unsafe fn, you must clearly document what the caller must guarantee. If your unsafe code is internal, you must document the invariants it assumes so maintainers can audit it later.

The best practice is to document invariants near the unsafe block. For example:

// SAFETY: ptr is aligned and valid for T, and the buffer lives for 'a

This makes the reasoning explicit and reviewable. You should also include an overall “Safety” section in your crate docs, summarizing invariants and design decisions.

Documentation also includes examples. A safe API should have examples that show correct usage and common pitfalls. This ties into tests: doctests can verify examples compile and run. A good documentation strategy is part of shipping a professional-grade crate.

How this fit on projects

This concept is applied in §10 (resources), §11 (self-assessment), and is essential for completing the capstone.

Definitions & key terms

  • Safety contract: The explicit invariants required for safety.
  • Unsafe fn: Function that requires caller to uphold invariants.
  • Doctest: Example in docs that is compiled and run.

Mental model diagram (ASCII)

Unsafe block
  |-- SAFETY: ... (invariants)
Docs -> users follow contract

How it works (step-by-step)

  1. Write invariants in docs.
  2. Add SAFETY comments near unsafe blocks.
  3. Add doctests for usage.

Minimal concrete example

/// # Safety
/// Caller must ensure ...
unsafe fn foo(...) { ... }

Common misconceptions

  • “Docs are optional.” They are part of the safety proof.

Check-your-understanding questions

  1. What should a SAFETY comment include?
  2. Why are doctests useful for unsafe APIs?

Check-your-understanding answers

  1. The exact invariant assumptions.
  2. They ensure examples and contracts are correct.

Real-world applications

  • Standard library unsafe functions.
  • Low-level crates (bytes, tokio, etc.).

Where you’ll apply it

References

  • Rustonomicon (unsafe guidelines).
  • Rust API Guidelines.

Key insights

Documentation is part of the correctness proof for unsafe code.

Summary

Safety contracts turn unsafe code into a maintainable, auditable system.

Homework/Exercises to practice the concept

  1. Add SAFETY comments to all unsafe blocks in previous projects.
  2. Write a Safety section for your crate.

Solutions to the homework/exercises

  1. Annotate each unsafe block with its invariants.
  2. Summarize invariants and misuse cases.

3. Project Specification

3.1 What You Will Build

A Rust crate of your choice that exposes a safe API built on unsafe internals. Examples: a fixed-capacity ring buffer, a slab allocator, a custom parser with zero-copy slices, or a small async runtime component.

3.2 Functional Requirements

  1. Public API is safe and ergonomic.
  2. Unsafe internals are documented with invariants.
  3. Test suite includes unit, property, and miri tests.
  4. CLI or example demo shows success and failure.

3.3 Non-Functional Requirements

  • Safety: No UB under documented usage.
  • Usability: Clear docs and examples.

3.4 Example Usage / Output

$ cargo run --example capstone_demo
success: all invariants upheld
exit code: 0

3.5 Data Formats / Schemas / Protocols

  • Depends on chosen domain; document explicitly (e.g., ring buffer layout).

3.6 Edge Cases

  • Invalid inputs.
  • Misuse attempts (should fail safely).

3.7 Real World Outcome

Deterministic demo and failure case.

3.7.1 How to Run (Copy/Paste)

cargo run --example capstone_demo

3.7.2 Golden Path Demo (Deterministic)

  • Fixed input and deterministic output.

3.7.3 CLI Transcript (Success)

$ cargo run --example capstone_demo
success: all invariants upheld
exit code: 0

3.7.4 Failure Demo (Invalid Input)

$ cargo run --example capstone_demo -- --bad-input
error: invalid argument
exit code: 2

4. Solution Architecture

4.1 High-Level Design

Safe API -> Unsafe core -> Tests + Docs

4.2 Key Components

Component Responsibility Key Decisions
Public API Encodes invariants lifetimes, newtypes
Unsafe core Performance-critical logic minimal surface
Tests Validate invariants miri + property
Docs Safety contracts SAFETY comments

4.4 Data Structures (No Full Code)

struct Core { /* unsafe internals */ }

4.4 Algorithm Overview

Describe the core algorithm of your chosen abstraction and analyze complexity.

5. Implementation Guide

5.1 Development Environment Setup

cargo new capstone
cd capstone
cargo add proptest

5.2 Project Structure

capstone/
├── src/
│   ├── lib.rs
│   └── core.rs
├── examples/
│   └── capstone_demo.rs
└── tests/
    └── prop_tests.rs

5.3 The Core Question You’re Answering

“How can I expose a safe API that makes misuse impossible, even with unsafe internals?”

5.4 Concepts You Must Understand First

  1. Invariant-driven API design.
  2. Unsafe contracts.
  3. Testing unsafe code.

5.5 Questions to Guide Your Design

  1. What invariants define safety for your abstraction?
  2. How will your API enforce them?
  3. What tests will catch violations?

5.6 Thinking Exercise

Write your API in a README before implementing anything.

5.7 The Interview Questions They’ll Ask

  1. “How do you prove unsafe code is safe?”
  2. “What is the invariant of your abstraction?”
  3. “How did you test for UB?”

5.8 Hints in Layers

Hint 1: Start with the safest possible API. Hint 2: Write invariants in comments before coding. Hint 3: Use miri early.

5.9 Books That Will Help

| Topic | Book | Chapter | |—|—|—| | Unsafe | Rustonomicon | Unsafe chapters | | API design | Rust API Guidelines | patterns |

5.10 Implementation Phases

Phase 1: API Design (1-2 weeks)

  • Define invariants and public types.

Phase 2: Unsafe Core (2-4 weeks)

  • Implement core logic with minimal unsafe.

Phase 3: Testing & Docs (2-4 weeks)

  • Add property tests, miri, and documentation.

5.11 Key Implementation Decisions

| Decision | Options | Recommendation | Rationale | |—|—|—|—| | Abstraction | allocator, ring buffer, parser | choose interest | motivation | | Testing | unit vs property | both | stronger safety | | Docs | minimal vs thorough | thorough | safety contract |

6. Testing Strategy

6.1 Test Categories

| Category | Purpose | Examples | |—|—|—| | Unit Tests | basic API | push/pop, alloc/free | | Property Tests | invariants | random sequences | | Miri Tests | UB detection | pointer operations |

6.2 Critical Test Cases

  1. Invariants hold for random sequences.
  2. Invalid input returns errors.
  3. No UB under miri.

6.3 Test Data

random sequences (seed=42)

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

| Pitfall | Symptom | Solution | |—|—|—| | Unclear invariants | unsafe bugs | write invariants first | | Too much unsafe | hard to audit | minimize unsafe | | Missing tests | latent UB | add property + miri |

7.2 Debugging Strategies

  • Use miri and loom if concurrency involved.

7.3 Performance Traps

  • Over-optimizing before correctness.

8. Extensions & Challenges

8.1 Beginner Extensions

  • Add a feature flag to toggle safety checks.

8.2 Intermediate Extensions

  • Add benchmarks and criterion reports.

8.3 Advanced Extensions

  • Publish the crate to crates.io with docs.

9. Real-World Connections

9.1 Industry Applications

  • Systems libraries with safe wrappers.
  • bytes, crossbeam, tokio.

9.3 Interview Relevance

  • Explain how you designed and proved safety.

10. Resources

10.1 Essential Reading

  • Rustonomicon.
  • Rust API Guidelines.

10.2 Video Resources

  • RustConf talks on unsafe Rust.

10.3 Tools & Documentation

  • miri, proptest, cargo-fuzz.

11. Self-Assessment Checklist

11.1 Understanding

  • I can articulate my invariants clearly.
  • I can explain why unsafe code is safe.

11.2 Implementation

  • Tests pass under miri.
  • Docs include Safety section.

11.3 Growth

  • I can present this project in interviews.

12. Submission / Completion Criteria

Minimum Viable Completion:

  • Safe API with documented invariants.
  • CLI demo with deterministic output and failure case.

Full Completion:

  • Property tests + miri validation.

Excellence (Going Above & Beyond):

  • Published crate with benchmarks and docs.