Project 4: Signal Laboratory

Build a set of programs that send, catch, and escalate signals for graceful shutdown.

Quick Reference

Attribute Value
Difficulty Intermediate
Time Estimate Weekend
Language C (Alternatives: Rust, Go, Python)
Prerequisites Basic C, process basics
Key Topics signals, handlers, graceful shutdown

1. Learning Objectives

By completing this project, you will:

  1. Install signal handlers with sigaction.
  2. Demonstrate SIGTERM vs SIGKILL behavior.
  3. Build a graceful shutdown sequence.
  4. Avoid common async-signal-safety bugs.

2. Theoretical Foundation

2.1 Core Concepts

  • Signals as interrupts: Signals are asynchronous events delivered by the kernel.
  • Signal disposition: Default, ignore, or custom handler; SIGKILL/SIGSTOP cannot be caught.
  • Async-signal safety: Only a small set of functions are safe inside handlers.

2.2 Why This Matters

Production daemons must shut down cleanly. Signal handling is the contract for safe termination.

2.3 Historical Context / Background

Signals date back to early Unix, providing a uniform method for process control and job management.

2.4 Common Misconceptions

  • “kill -9 is normal”: It bypasses cleanup and should be a last resort.
  • “printf in handlers is fine”: It is not async-signal-safe.

3. Project Specification

3.1 What You Will Build

A small suite: a signal receiver with handlers, a sender tool, and a manager that gracefully shuts down child workers.

3.2 Functional Requirements

  1. Register handlers for SIGINT, SIGTERM, SIGHUP.
  2. Demonstrate SIGKILL and SIGSTOP cannot be handled.
  3. Implement a graceful shutdown flow for children.

3.3 Non-Functional Requirements

  • Reliability: Cleanup happens outside handlers.
  • Safety: No unsafe calls in signal handlers.
  • Clarity: Logs show signal flow.

3.4 Example Usage / Output

$ ./signal-receiver
[PID 1234] Ready
$ kill -SIGTERM 1234
[PID 1234] SIGTERM received, shutting down...

3.5 Real World Outcome

You will send signals to your receiver and see clean shutdown logs and proper child cleanup. Example:

$ ./signal-receiver
[PID 1234] Ready
$ kill -SIGTERM 1234
[PID 1234] SIGTERM received, shutting down...

4. Solution Architecture

4.1 High-Level Design

Sender -> kernel delivers signal -> handler sets flag -> main loop cleans up

4.2 Key Components

Component Responsibility Key Decisions
Receiver Handle signals Use sigaction
Manager Spawn/stop workers SIGTERM then SIGKILL
Sender Send signals Use kill(2)

4.3 Data Structures

volatile sig_atomic_t shutdown_requested = 0;

4.4 Algorithm Overview

Key Algorithm: Graceful Shutdown

  1. Handler sets a flag.
  2. Main loop detects flag.
  3. Stop workers and wait.

Complexity Analysis:

  • Time: O(n) workers
  • Space: O(n) PIDs

5. Implementation Guide

5.1 Development Environment Setup

gcc --version

5.2 Project Structure

project-root/
├── receiver.c
├── manager.c
├── sender.c
└── README.md

5.3 The Core Question You’re Answering

“How do I shut down a process safely when the OS tells it to stop?”

5.4 Concepts You Must Understand First

Stop and research these before coding:

  1. Signal Disposition
    • Default actions and which signals cannot be caught.
  2. sigaction
    • Using sa_handler, sa_flags, and sigemptyset.
  3. SIGCHLD
    • Reaping children to avoid zombies.

5.5 Questions to Guide Your Design

Before implementing, think through these:

  1. What cleanup is safe to do inside the handler?
  2. How long should a worker have to exit before SIGKILL?
  3. How will you confirm a worker exited?

5.6 Thinking Exercise

Write a Safe Handler

Write a handler that only sets a flag. Move real cleanup into the main loop.

5.7 The Interview Questions They’ll Ask

Prepare to answer these:

  1. “Why is SIGKILL different from SIGTERM?”
  2. “Why is printf unsafe in a signal handler?”
  3. “How do you prevent zombie processes?”

5.8 Hints in Layers

Hint 1: Use sigaction It is more reliable than signal().

Hint 2: Use a flag Set a volatile sig_atomic_t flag and exit cleanly in the main loop.

Hint 3: Reap children Use waitpid with WNOHANG.

5.9 Books That Will Help

Topic Book Chapter
Signals “TLPI” Ch. 20-22
Process control “APUE” Ch. 10
Daemon patterns “Linux System Programming” Ch. 6

5.10 Implementation Phases

Phase 1: Foundation (1 day)

Goals:

  • Catch SIGINT and SIGTERM.

Tasks:

  1. Create receiver with sigaction.
  2. Print a message on shutdown.

Checkpoint: Receiver exits cleanly on SIGTERM.

Phase 2: Core Functionality (1 day)

Goals:

  • Manage workers.

Tasks:

  1. Spawn child processes.
  2. Send SIGTERM then SIGKILL if needed.

Checkpoint: Workers exit and are reaped.

Phase 3: Polish & Edge Cases (Half day)

Goals:

  • Improve logs and handle SIGCHLD.

Tasks:

  1. Add SIGCHLD handler.
  2. Summarize shutdown results.

Checkpoint: No zombies after shutdown.

5.11 Key Implementation Decisions

Decision Options Recommendation Rationale
Handler API signal vs sigaction sigaction Portable and robust
Escalation immediate kill vs grace grace then kill Safe cleanup

6. Testing Strategy

6.1 Test Categories

Category Purpose Examples
Handler Catch signals SIGINT, SIGTERM
Escalation SIGKILL fallback Unresponsive worker
Reaping No zombies ps -o stat

6.2 Critical Test Cases

  1. SIGTERM triggers graceful shutdown.
  2. SIGKILL stops a stuck worker.
  3. No zombies remain after exit.

6.3 Test Data

Worker PIDs list

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

Pitfall Symptom Solution
Unsafe handler calls Deadlocks/crashes Only set flags
Ignoring SIGCHLD Zombies Call waitpid
Short timeouts Forced kills Increase grace period

7.2 Debugging Strategies

  • Use strace -e signal to view signal delivery.
  • Inspect process states with ps -o pid,ppid,stat,cmd.

7.3 Performance Traps

Overly aggressive signal loops can spin; use sleeps between checks.

8. Extensions & Challenges

8.1 Beginner Extensions

  • Add SIGHUP reload behavior.
  • Add configurable timeouts.

8.2 Intermediate Extensions

  • Add a supervisor that restarts crashed workers.
  • Add signal masking for critical sections.

8.3 Advanced Extensions

  • Implement a small daemon with PID file handling.
  • Integrate with systemd service unit signals.

9. Real-World Connections

9.1 Industry Applications

  • Graceful shutdown is required for systemd, Kubernetes, and service managers.
  • systemd: https://systemd.io
  • supervisord: http://supervisord.org

9.3 Interview Relevance

  • Signals and process cleanup are core Unix topics.

10. Resources

10.1 Essential Reading

  • signal(7) - man 7 signal
  • sigaction(2) - man 2 sigaction

10.2 Video Resources

  • Unix signals overview (search “Unix signals sigaction”)

10.3 Tools & Documentation

  • kill(1) - man 1 kill

11. Self-Assessment Checklist

11.1 Understanding

  • I can explain signal delivery.
  • I can describe SIGTERM vs SIGKILL.
  • I can explain async-signal safety.

11.2 Implementation

  • Handlers are installed with sigaction.
  • Shutdown is graceful and reliable.
  • Children are reaped.

11.3 Growth

  • I can apply this to a production daemon.
  • I can explain shutdown semantics in interviews.

12. Submission / Completion Criteria

Minimum Viable Completion:

  • Signal receiver handles SIGINT/SIGTERM safely.

Full Completion:

  • Manager shuts down child processes gracefully.

Excellence (Going Above & Beyond):

  • Add reload behavior and integrate with service manager patterns.

This guide was generated from LINUX_SYSTEM_TOOLS_MASTERY.md. For the complete learning path, see the parent directory.