Project 3: Interrupt Storm - Latency and Priority Explorer

Build an interrupt latency benchmark that measures best and worst-case response times under load.

Quick Reference

Attribute Value
Difficulty Level 2: Intermediate
Time Estimate 10-15 hours
Main Programming Language C++
Alternative Programming Languages C, Rust
Coolness Level Level 3: Genuinely Clever
Business Potential 1. The “Resume Gold”
Prerequisites C/C++ basics, Teensyduino setup, basic electronics, ability to use a multimeter/logic analyzer
Key Topics NVIC priorities, ISR latency, critical sections, jitter

1. Learning Objectives

By completing this project, you will:

  1. Explain the core question for this project in your own words.
  2. Implement the main workflow and validate it with measurements.
  3. Handle at least two failure modes and document recovery.
  4. Produce a deterministic report that matches hardware behavior.

2. All Theory Needed (Per-Concept Breakdown)

Interrupt Latency, Priorities, and Critical Sections

Fundamentals Interrupts are hardware events that suspend the main program and run an ISR. The NVIC manages priorities and preemption. Latency is the time between event and ISR execution, and it is affected by critical sections, disabled interrupts, and higher-priority ISRs. Measuring latency is the only way to know if a real-time constraint is truly met.

Deep Dive into the concept When an interrupt fires, the Cortex-M hardware saves state and vectors to the ISR. The entry cost is fixed, but only if interrupts are enabled and no higher priority interrupt is running. Long critical sections extend worst-case latency and can cause missed deadlines. Priority inversion happens when a low-priority task or ISR blocks a higher priority one, often through shared resources. In real systems multiple sources compete: timers, GPIO edges, DMA completion, USB, and more. Priorities let you preserve critical response times, but the cost is more jitter in lower-priority tasks. A reliable design keeps ISRs short, defers work to tasks or the main loop, and uses ring buffers for data capture. Measuring latency requires a reference event and a response. Toggle a GPIO at ISR entry and measure the time between the stimulus edge and response edge. Collect hundreds or thousands of samples to measure distribution and jitter. This project builds a latency report so you can quantify real-time behavior under load.

How this fit on projects This concept directly powers the implementation choices and validation steps in this project.

Definitions & key terms

  • NVIC: Nested Vectored Interrupt Controller that manages priority and preemption.
  • ISR: Interrupt Service Routine executed in response to an interrupt.
  • Latency: Time between event and ISR start.
  • Critical section: Code region where interrupts are disabled.

Mental model diagram (ASCII)

Event -> NVIC -> ISR -> GPIO toggle

How it works (step-by-step)

  1. Configure two interrupt sources with different priorities.
  2. Toggle a GPIO at ISR entry and exit.
  3. Generate load and record timing on a logic analyzer.
  4. Compute best/average/worst latency.

Minimal concrete example

void isr() { digitalWriteFast(5, HIGH); digitalWriteFast(5, LOW); }
attachInterrupt(digitalPinToInterrupt(2), isr, RISING);

Common misconceptions

  • ISRs can do long work safely.
  • Disabling interrupts has negligible cost.
  • Latency is a fixed constant.

Check-your-understanding questions

  • What increases worst-case latency?
  • Why should ISRs avoid dynamic allocation?
  • How does priority inversion happen?

Check-your-understanding answers

  • Long critical sections, higher priority ISRs, or disabled interrupts.
  • Allocators are slow and can be non-deterministic.
  • A low-priority ISR or task holds a resource needed by a high-priority one.

Real-world applications

  • Servo control
  • Real-time audio processing
  • Industrial control loops

Where you’ll apply it

References

  • ARM Cortex-M7 Technical Reference Manual
  • Making Embedded Systems, Ch. 5

Key insights

Measure latency under realistic load or you will overestimate capability.

Summary

Interrupt priorities and critical sections determine real-time responsiveness.

Homework/Exercises to practice the concept

  • Measure ISR latency with interrupts disabled for 100 us blocks.
  • Plot a histogram of 1000 latency samples.

Solutions to the homework/exercises

  • Use a timer ISR to generate events and measure response on a GPIO.
  • Export timestamps to CSV and compute percentile latency.

3. Project Specification

3.1 What You Will Build

Build an interrupt latency benchmark that measures best and worst-case response times under load.

3.2 Functional Requirements

  1. Generate two competing interrupt sources.
  2. Measure latency distribution under idle and load.
  3. Detect and report priority inversion scenarios.
  4. Export latency histogram to CSV.

3.3 Non-Functional Requirements

  • Performance: Meet the target timing/throughput for the project.
  • Reliability: Detect errors and recover without undefined behavior.
  • Usability: Provide clear logs and a repeatable workflow.

3.4 Example Usage / Output

./P03-interrupt-storm-latency-and-priority-explorer --run

3.5 Data Formats / Schemas / Protocols

CSV with columns: sample_id, latency_us, mode

3.6 Edge Cases

  • Interrupts disabled in critical sections
  • DMA bursts causing latency spikes
  • ISR too long causing starvation

3.7 Real World Outcome

You will run the project and see deterministic logs and measurements that match physical hardware behavior.

3.7.1 How to Run (Copy/Paste)

cd project-root
make
./P03-interrupt-storm-latency-and-priority-explorer --run

3.7.2 Golden Path Demo (Deterministic)

Use a fixed input configuration and a known test signal. Capture output for 60 seconds and verify it matches expected values.

3.7.3 If CLI: exact terminal transcript

$ ./P03-interrupt-storm-latency-and-priority-explorer --run --seed 42
[INFO] Interrupt Storm - Latency and Priority Explorer starting
[INFO] Report saved to data/report.csv
[INFO] Status: OK
$ echo $?
0

Failure Demo (Deterministic)

$ ./P03-interrupt-storm-latency-and-priority-explorer --run --missing-device
[ERROR] Device not detected
$ echo $?
2

4. Solution Architecture

4.1 High-Level Design

Inputs -> Acquisition -> Processing -> Output/Log

4.2 Key Components

Component Responsibility Key Decisions
Acquisition Configure peripherals and capture data Use stable clock settings
Processing Convert raw data to meaningful values Apply calibration/filters
Output/Log Emit reports and logs CSV for reproducibility

4.3 Data Structures (No Full Code)

struct Sample {
    uint32_t timestamp_us;
    uint32_t value;
    uint32_t flags;
};

4.4 Algorithm Overview

Key Algorithm: Measurement + Report

  1. Initialize hardware and verify configuration.
  2. Capture data and record timestamps.
  3. Compute metrics and write report.

Complexity Analysis:

  • Time: O(n) in samples
  • Space: O(n) for log storage

5. Implementation Guide

5.1 Development Environment Setup

# Arduino IDE + Teensyduino must be installed
# Optional CLI workflow
arduino-cli core update-index
arduino-cli core install teensy:avr

5.2 Project Structure

project-root/
├── src/
│   ├── main.ino
│   ├── hw_config.h
│   └── measurements.cpp
├── tools/
│   └── analyze.py
├── data/
│   └── samples.csv
└── README.md

5.3 The Core Question You’re Answering

“How fast can my system react when multiple events compete for attention?”

5.4 Concepts You Must Understand First

Stop and research these before coding:

  1. NVIC priorities, ISR latency, critical sections, jitter
  2. Data logging and measurement techniques
  3. Basic timing math and error analysis

5.5 Questions to Guide Your Design

  1. Which GPIO will represent ISR entry?
  2. How will you generate controlled load?
  3. How many samples are enough to measure worst-case?

5.6 Thinking Exercise

Assign priorities to three interrupts and predict execution order.

5.7 The Interview Questions They’ll Ask

  1. What is interrupt latency?
  2. Why should ISRs be short?
  3. How does priority inversion happen?

5.8 Hints in Layers

  • Toggle a pin at ISR entry and exit.
  • Use a timer ISR as load.
  • Disable DMA to see baseline latency.

5.9 Books That Will Help

| Topic | Book | Chapter | |——-|——|———| | Interrupt design | Making Embedded Systems | Ch. 5 | | Concurrency safety | Effective C | Ch. 7 | | Debugging | The Art of Debugging | Ch. 2 |

5.10 Implementation Phases

Phase 1: Foundation (4 hours)

Goals:

  • Configure two interrupts
  • Build measurement GPIO

Tasks:

  1. Configure two interrupts
  2. Build measurement GPIO

Checkpoint: Latency visible on analyzer

Phase 2: Core Functionality (6 hours)

Goals:

  • Load system with DMA/CPU
  • Log latency data

Tasks:

  1. Load system with DMA/CPU
  2. Log latency data

Checkpoint: Latency histogram

Phase 3: Polish (3 hours)

Goals:

  • Analyze priority inversion
  • Document results

Tasks:

  1. Analyze priority inversion
  2. Document results

Checkpoint: Final report

5.11 Key Implementation Decisions

| Decision | Options | Recommendation | Rationale | |———-|———|—————-|———–| | Buffering | Single buffer, double buffer | Double buffer | Avoids data loss during processing | | Logging format | CSV, binary | CSV | Human-readable while still scriptable | | Clock speed | Default, overclock | Default | Keeps peripherals in spec |

6. Testing Strategy

6.1 Test Categories

| Category | Purpose | Examples | |———-|———|———-| | Unit Tests | Validate math, parsing, and conversions | Timer math, CRC checks | | Integration Tests | Verify peripherals and pipelines | DMA -> buffer -> log | | Edge Case Tests | Handle boundary conditions | Brownout, missing sensor |

6.2 Critical Test Cases

{test_cases}

6.3 Test Data

Use a fixed test input pattern and record outputs to data/report.csv

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

| Pitfall | Symptom | Solution | |———|———|———-| {pitfalls}

7.2 Debugging Strategies

{debug_strats}

7.3 Performance Traps

Large buffers improve stability but increase latency. Measure both throughput and jitter to choose the right size.

8. Extensions & Challenges

8.1 Beginner Extensions

{ex_begin}

8.2 Intermediate Extensions

{ex_inter}

8.3 Advanced Extensions

{ex_adv}

9. Real-World Connections

9.1 Industry Applications

{industry_apps}

{open_source}

9.3 Interview Relevance

{interview_rel}

10. Resources

10.1 Essential Reading

{resources}

10.2 Video Resources

  • Embedded systems timing walkthrough (YouTube)
  • Teensy hardware deep dive (Conference talk)

10.3 Tools & Documentation

  • Teensyduino: Toolchain for Teensy boards
  • Logic Analyzer: Timing verification
  • Multimeter: Voltage and current measurement

{related_projects}

11. Self-Assessment Checklist

11.1 Understanding

  • I can explain the main concept without notes.
  • I can explain why the measurements match (or do not match) expectations.
  • I understand at least one tradeoff made in this project.

11.2 Implementation

  • All functional requirements are met.
  • All critical test cases pass.
  • Logs and reports are reproducible.
  • Edge cases are handled.

11.3 Growth

  • I documented lessons learned.
  • I can explain this project in a job interview.
  • I identified one improvement for next iteration.

12. Submission / Completion Criteria

Minimum Viable Completion: {comp_min}

Full Completion: {comp_full}

Excellence (Going Above & Beyond): {comp_ex}