Build a C Debugger from Scratch: From Zero to Mini-GDB
Goal: Deeply understand how debuggers work at the lowest level by building one from scratch. You will implement process attachment, memory inspection, register manipulation, breakpoints, single-stepping, signal handling, DWARF parsing, source-level debugging, stack unwinding, watchpoints, expression evaluation, and multi-threaded debugging. By the end, you will have a functional mini-GDB for Linux and will understand the kernel interfaces, binary formats, and CPU mechanics that make debugging possible.
Why Building a Debugger Matters
Debuggers are the most powerful tools in a systems programmer’s arsenal, yet most developers treat them as black boxes. Building a debugger from scratch is the ultimate exercise in systems programming because it forces you to understand:
- How the OS controls processes: The
ptracesystem call is the kernel’s debugging API - How CPUs execute code: Breakpoints, single-stepping, and register state
- How binaries are structured: ELF format, symbol tables, and DWARF debug info
- How source maps to assembly: Line number tables and variable locations
- How the stack works: Call frames, unwinding, and parameter passing
The Debugging Triangle (All Three Must Be Mastered)
┌─────────────────┐
│ Source Code │
│ (DWARF) │
└────────▲────────┘
│
Line tables │ Location info
│
┌────────────────────┼────────────────────┐
│ │ │
▼ │ ▼
┌───────────────┐ │ ┌───────────────┐
│ Process │◄───────────┴───────────►│ Binary │
│ (ptrace) │ Memory mapping │ (ELF) │
└───────────────┘ └───────────────┘
│ │
│ Registers, memory, signals │ Symbols, sections
│ │
└───────────────────┬───────────────────────┘
│
▼
┌─────────────────┐
│ DEBUGGER │
│ (What you build)│
└─────────────────┘
What Professional Debugger Developers Know
When you complete this guide, you will understand:
- The ptrace API: How one process controls another at the kernel level
- Breakpoint mechanics: Why
INT3(0xCC) is special and how software breakpoints work - CPU state: How to read/write registers, including the instruction pointer
- ELF format: Sections, symbols, and how to find function addresses
- DWARF debug info: Compilation units, line tables, and variable locations
- Signal handling: How
SIGTRAPandSIGSTOPenable debugging - Stack unwinding: How to walk the call stack with frame pointers or CFI
- Memory inspection: Reading and writing another process’s address space
- Watchpoints: Using hardware debug registers for data breakpoints
- Expression evaluation: Parsing and evaluating C expressions in context
The Numbers Behind Debuggers
- GDB: 1.5+ million lines of code, started in 1986
- LLDB: 800K+ lines, modern architecture with LLVM integration
- ptrace operations: 20+ distinct request types on Linux
- DWARF sections: 10+ different
.debug_*sections in a typical binary - x86-64 registers: 16 general-purpose + debug registers + SSE/AVX
Who This Guide Is For
Your Background What You'll Gain
──────────────────────────────────────────────────────────────
Debugger user Deep understanding of what happens
(use GDB/LLDB daily) when you type "break main"
Systems programmer The missing piece: how tools like
(C, assembly, OS concepts) strace, gdb, valgrind actually work
Reverse engineer Foundation for building analysis
(binary analysis) tools, tracers, and instrumentation
Compiler developer Understanding how debug info is
(LLVM, GCC internals) consumed and what matters to emit
Prerequisites & Background Knowledge
Essential Prerequisites (Must Have)
C Programming (Intermediate+):
- Pointers, pointer arithmetic, and casting
- System calls (
fork,exec,wait,open,read,write) - Signal handling basics (
signal,sigaction) - Memory layout (stack, heap, code, data segments)
- Recommended Reading: “The C Programming Language” (K&R) or “C Programming: A Modern Approach” (King)
Linux/UNIX Fundamentals:
- Process creation and lifecycle
- Virtual memory concepts
- File descriptors and
/procfilesystem - Basic command line and
make - Recommended Reading: “The Linux Programming Interface” (Kerrisk) Ch. 1-6
Computer Architecture Basics:
- Registers and the instruction pointer
- Stack frames and calling conventions
- Instruction encoding basics
- Recommended Reading: “Computer Systems: A Programmer’s Perspective” (CS:APP) Ch. 3
Helpful But Not Required
x86-64 Assembly:
- Basic instruction set (mov, push, pop, call, ret)
- Calling conventions (System V AMD64 ABI)
- Can learn during: Projects 5, 6, 7
ELF Binary Format:
- Section headers, symbol tables
- Dynamic linking basics
- Can learn during: Projects 10, 11
DWARF Debug Format:
- Will be covered in depth during Projects 10-12
- Recommended Reading: “The DWARF Debugging Standard”
Self-Assessment Questions
Before starting, you should be able to answer:
- What happens when you call
fork()? What does the child process inherit? - How do you compile a C program with debug symbols? What flag do you use?
- What is the
/proc/<pid>/mapsfile? What information does it contain? - What is the difference between the stack pointer (RSP) and instruction pointer (RIP)?
- What is a system call? How is it different from a function call?
- What happens when a program receives a signal like
SIGSEGV?
If you cannot answer 4+ of these questions, spend time with “The Linux Programming Interface” Ch. 1-6 and 20-22 first.
Development Environment Setup
Required:
- Linux system (x86-64) - Ubuntu 22.04+ or similar
- GCC or Clang with debug symbol support
- Make or CMake build system
- GDB (for comparison and verification)
Recommended Tools:
# Install development tools
sudo apt install build-essential gdb
# Install ELF/DWARF tools
sudo apt install binutils elfutils libdwarf-dev libelf-dev
# Install strace for tracing system calls
sudo apt install strace
# Verify setup
gcc --version
gdb --version
readelf --version
Testing Your Setup:
# Create a simple test program
cat > test.c << 'EOF'
#include <stdio.h>
int main() {
int x = 42;
printf("x = %d\n", x);
return 0;
}
EOF
# Compile with debug info
gcc -g -O0 -o test test.c
# Verify debug info exists
readelf --debug-dump=info test | head -20
# Verify ptrace works (requires permissions)
strace -e ptrace gdb -batch -ex "break main" -ex "run" -ex "quit" ./test
Time Investment
| Project Group | Projects | Time Estimate |
|---|---|---|
| Foundation (ptrace basics) | 1-4 | 2-3 weeks |
| Core Debugging (breakpoints, stepping) | 5-8 | 3-4 weeks |
| Debug Info (ELF/DWARF) | 9-12 | 4-5 weeks |
| Advanced Features | 13-16 | 4-6 weeks |
| Total | 16 | 13-18 weeks |
Important Reality Check
Building a debugger is hard. You will:
- Read kernel documentation and header files
- Parse complex binary formats (ELF, DWARF)
- Debug your debugger with another debugger
- Encounter platform-specific behavior
- Need patience with cryptic error messages
The reward is understanding the deepest layer of software development tooling.
Core Concept Analysis
1. The ptrace System Call
ptrace (process trace) is the Linux kernel interface that enables debugging. It allows one process (the tracer/debugger) to observe and control another process (the tracee/debuggee).
ptrace Architecture:
┌─────────────────────────────────────────────────────────────────────┐
│ KERNEL SPACE │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ ptrace SUBSYSTEM │ │
│ │ │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
│ │ │ ATTACH │ │ PEEK │ │ POKE │ │ │
│ │ │ PTRACE_ATTACH│ │ PTRACE_PEEKTEXT │ PTRACE_POKETEXT │ │
│ │ │ PTRACE_SEIZE │ │ PTRACE_PEEKDATA │ PTRACE_POKEDATA │ │
│ │ │ PTRACE_TRACEME │ PTRACE_PEEKUSER │ PTRACE_POKEUSER │ │
│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │
│ │ │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
│ │ │ CONTROL │ │ REGISTERS │ │ STEP │ │ │
│ │ │ PTRACE_CONT │ │ PTRACE_GETREGS │ PTRACE_SINGLESTEP│ │
│ │ │ PTRACE_DETACH│ │ PTRACE_SETREGS │ PTRACE_SYSCALL │ │
│ │ │ PTRACE_KILL │ │ PTRACE_GETFPREGS │ │ │
│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │ │
│ │ signals (SIGTRAP, SIGSTOP, etc.) │
│ ▼ │
├─────────────────────────────────────────────────────────────────────┤
│ USER SPACE │
│ │
│ ┌─────────────────┐ ┌─────────────────┐ │
│ │ DEBUGGER │◄─────────►│ DEBUGGEE │ │
│ │ (tracer) │ ptrace │ (tracee) │ │
│ │ │ calls │ │ │
│ │ - Sets BP │ │ - Runs code │ │
│ │ - Reads mem │ │ - Stops at BP │ │
│ │ - Reads regs │ │ - Receives │ │
│ │ - Single steps │ │ signals │ │
│ └─────────────────┘ └─────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘
Key ptrace Operations:
| Operation | Purpose | When Used |
|---|---|---|
PTRACE_TRACEME |
Child requests tracing by parent | Launch debuggee |
PTRACE_ATTACH |
Attach to running process | Attach to PID |
PTRACE_PEEKTEXT |
Read word from code segment | Read instructions |
PTRACE_POKETEXT |
Write word to code segment | Set breakpoints |
PTRACE_GETREGS |
Get all general registers | Inspect state |
PTRACE_SETREGS |
Set all general registers | Modify state |
PTRACE_SINGLESTEP |
Execute one instruction | Step command |
PTRACE_CONT |
Continue execution | Continue command |
2. Software Breakpoints (INT3)
Software breakpoints work by replacing an instruction with a trap instruction. On x86, this is INT3 (opcode 0xCC), which generates a SIGTRAP signal.
Breakpoint Lifecycle:
1. ORIGINAL CODE:
0x401000: 48 89 e5 mov rbp, rsp ◄── target address
0x401003: 48 83 ec 10 sub rsp, 0x10
0x401007: ...
2. SET BREAKPOINT (save original byte, write 0xCC):
0x401000: CC int3 ◄── TRAP!
0x401001: 89 e5 (corrupted)
0x401003: 48 83 ec 10 sub rsp, 0x10
3. CPU HITS INT3:
- CPU generates trap exception
- Kernel delivers SIGTRAP to tracee
- Tracee stops, tracer notified via wait()
- RIP points to 0x401001 (one past INT3)
4. DEBUGGER HANDLES BREAKPOINT:
a. Restore original byte (0x48)
b. Decrement RIP by 1 (back to 0x401000)
c. Single-step to execute original instruction
d. Re-insert 0xCC for next hit
e. Continue execution
Why INT3?
- Single-byte instruction (doesn’t break instruction boundaries)
- Specifically designed for debuggers
- Generates precise trap (synchronous exception)
- CPU saves exact state at trap point
3. x86-64 Registers and Debugging
The CPU’s register state is central to debugging. On x86-64, you need to know:
x86-64 General Purpose Registers:
┌─────────────────────────────────────────────────────────────────────┐
│ 64-bit 32-bit 16-bit 8-bit │
├─────────────────────────────────────────────────────────────────────┤
│ Accumulator RAX EAX AX AL/AH │
│ Base RBX EBX BX BL/BH │
│ Counter RCX ECX CX CL/CH │
│ Data RDX EDX DX DL/DH │
│ Source Index RSI ESI SI SIL │
│ Dest Index RDI EDI DI DIL │
│ Base Pointer RBP EBP BP BPL │
│ Stack Pointer RSP ESP SP SPL │
│ Extended R8-R15 R8D-R15D R8W-R15W R8B-R15B │
│ │
│ Instruction Ptr RIP EIP IP - │
│ Flags RFLAGS EFLAGS FLAGS - │
└─────────────────────────────────────────────────────────────────────┘
System V AMD64 ABI Calling Convention:
┌───────┬────────────────────────────────────────────────────────────┐
│ RDI │ First argument │
│ RSI │ Second argument │
│ RDX │ Third argument │
│ RCX │ Fourth argument │
│ R8 │ Fifth argument │
│ R9 │ Sixth argument │
│ Stack │ Seventh+ arguments (pushed right-to-left) │
├───────┼────────────────────────────────────────────────────────────┤
│ RAX │ Return value │
│ RDX │ Second return value (for 128-bit returns) │
├───────┼────────────────────────────────────────────────────────────┤
│ RBX │ Callee-saved (preserved across calls) │
│ RBP │ Callee-saved / frame pointer │
│ R12-15│ Callee-saved │
└───────┴────────────────────────────────────────────────────────────┘
Debug Registers (DR0-DR7):
Hardware Watchpoint Registers:
DR0-DR3: Linear address breakpoint registers
Store addresses to watch for read/write/execute
DR6: Debug status register
- B0-B3: Which DR0-DR3 triggered
- BS: Single-step trap
- BD: Debug register access
DR7: Debug control register
- L0-L3: Local enable for DR0-DR3
- G0-G3: Global enable for DR0-DR3
- RW0-RW3: Condition (00=exec, 01=write, 11=read/write)
- LEN0-LEN3: Size (00=1, 01=2, 10=8, 11=4 bytes)
4. ELF Binary Format
Understanding ELF (Executable and Linkable Format) is essential for finding function addresses, symbols, and debug information.
ELF File Structure:
┌──────────────────────────────────────┐
│ ELF Header │ Magic: 7f 45 4c 46 (0x7f "ELF")
│ - Magic number, class (32/64-bit) │ Entry point, program header offset
│ - Entry point address │ Section header offset
│ - Program/Section header offsets │
├──────────────────────────────────────┤
│ Program Headers │ Describe segments for loading
│ - PT_LOAD (code, data) │ Virtual addresses, permissions
│ - PT_DYNAMIC (dynamic linking) │
│ - PT_INTERP (interpreter path) │
├──────────────────────────────────────┤
│ │
│ Sections │
│ │
│ .text - Executable code │
│ .data - Initialized data │
│ .bss - Uninitialized data │
│ .rodata - Read-only data │
│ .symtab - Symbol table │
│ .strtab - String table │
│ .dynsym - Dynamic symbols │
│ │
│ .debug_info - DWARF DIEs │
│ .debug_abbrev - Abbreviations │
│ .debug_line - Line number table │
│ .debug_frame - Call frame info │
│ .debug_str - Debug strings │
│ │
├──────────────────────────────────────┤
│ Section Headers │ Describe each section
│ - Name, type, flags │ Offset, size, alignment
│ - Address, offset, size │
└──────────────────────────────────────┘
Key ELF Concepts for Debuggers:
| Concept | Section | Debugger Use |
|---|---|---|
| Function addresses | .symtab, .dynsym |
break function_name |
| Source file mapping | .debug_info |
list file.c:42 |
| Line numbers | .debug_line |
break file.c:42 |
| Variable locations | .debug_info |
print variable |
| Call frame info | .debug_frame |
backtrace |
| Type information | .debug_info |
print struct_ptr->field |
5. DWARF Debug Format
DWARF (Debugging With Attributed Record Formats) is the standard debug information format for ELF binaries.
DWARF Structure Overview:
.debug_info (DIEs - Debugging Information Entries):
┌────────────────────────────────────────────────────────────────────┐
│ Compilation Unit │
│ DW_TAG_compile_unit │
│ ├── name: "main.c" │
│ ├── comp_dir: "/home/user/project" │
│ ├── low_pc: 0x401000 │
│ ├── high_pc: 0x401200 │
│ │ │
│ ├── DW_TAG_subprogram (function) │
│ │ ├── name: "main" │
│ │ ├── low_pc: 0x401100 │
│ │ ├── high_pc: 0x401180 │
│ │ ├── frame_base: DW_OP_call_frame_cfa │
│ │ │ │
│ │ ├── DW_TAG_formal_parameter │
│ │ │ ├── name: "argc" │
│ │ │ ├── type: → int │
│ │ │ └── location: DW_OP_fbreg -20 │
│ │ │ │
│ │ └── DW_TAG_variable │
│ │ ├── name: "x" │
│ │ ├── type: → int │
│ │ └── location: DW_OP_fbreg -24 │
│ │ │
│ └── DW_TAG_base_type │
│ ├── name: "int" │
│ ├── byte_size: 4 │
│ └── encoding: DW_ATE_signed │
└────────────────────────────────────────────────────────────────────┘
.debug_line (Line Number Table):
┌────────────────────────────────────────────────────────────────────┐
│ Address File Line Column Statement │
├────────────────────────────────────────────────────────────────────┤
│ 0x401100 main.c 5 0 true │
│ 0x401108 main.c 6 4 true │
│ 0x401115 main.c 7 4 true │
│ 0x401120 main.c 8 8 true │
│ 0x401135 main.c 9 4 true │
│ 0x401150 main.c 10 0 true │
└────────────────────────────────────────────────────────────────────┘
.debug_frame (Call Frame Information):
┌────────────────────────────────────────────────────────────────────┐
│ CIE (Common Information Entry) │
│ - Code alignment factor │
│ - Data alignment factor │
│ - Return address register │
│ - Initial instructions (for unwinding) │
│ │
│ FDE (Frame Description Entry) for each function │
│ - Initial location, address range │
│ - Instructions to unwind at each PC │
│ - CFA = RSP + 8 (after call) │
│ - CFA = RBP + 16 (after push rbp; mov rbp, rsp) │
│ - Return address at CFA - 8 │
└────────────────────────────────────────────────────────────────────┘
6. Stack Unwinding
Stack unwinding is essential for backtrace functionality. There are two main approaches:
Stack Frame Layout (x86-64 with frame pointers):
High addresses
│
▼
┌───────────────────┐
│ main's locals │
├───────────────────┤
│ Return addr │◄── Where main returns to (_start)
├───────────────────┤
│ Saved RBP │◄── main's RBP points here
├───────────────────┤
│ foo's args │ (if > 6 args)
├───────────────────┤
│ foo's locals │
├───────────────────┤
│ Return addr │◄── Where foo returns to (main)
├───────────────────┤
│ Saved RBP │◄── foo's RBP points here
├───────────────────┤
│ bar's args │
├───────────────────┤
│ bar's locals │
├───────────────────┤◄── RSP (current stack pointer)
│
▼
Low addresses
Unwinding with Frame Pointers:
1. Start at current RBP
2. Saved RBP = *(RBP) → previous frame's RBP
3. Return addr = *(RBP+8) → caller's next instruction
4. Repeat until RBP = 0 or invalid
Unwinding with DWARF CFI (no frame pointer):
1. Lookup current RIP in .debug_frame
2. Apply CFI rules to compute CFA (Canonical Frame Address)
3. CFA typically = RSP + offset
4. Return address at CFA - 8
5. Restore callee-saved registers per CFI
6. Repeat for each frame
Concept Summary Table
| Concept | What You Must Internalize |
|---|---|
| ptrace | The kernel API for process control; attach, peek, poke, step, continue |
| Breakpoints | Replace instruction with INT3 (0xCC), handle SIGTRAP, restore and re-insert |
| Registers | RIP is instruction pointer, RSP is stack, RBP is frame pointer; use GETREGS/SETREGS |
| Memory access | PEEKDATA/POKEDATA work word-at-a-time; use /proc/ |
| Signals | SIGTRAP for breakpoints, SIGSTOP for pausing; waitpid returns status |
| ELF format | Sections contain code/data/symbols; .symtab maps names to addresses |
| DWARF | .debug_info has types/functions, .debug_line maps addresses to source |
| Stack unwinding | Follow RBP chain or use CFI data; each frame has return address |
| Watchpoints | Hardware debug registers DR0-DR3; limited to 4 simultaneous watches |
Deep Dive Reading by Concept
| Concept | Book & Chapter |
|---|---|
| ptrace fundamentals | “The Linux Programming Interface” by Kerrisk — Ch. 24 (Process Creation) + man ptrace |
| Breakpoint mechanics | “How Debuggers Work” by Eli Bendersky — Blog series |
| x86 architecture | “CS:APP 3rd Edition” — Ch. 3 (Machine-Level Representation) |
| ELF format | “Linkers and Loaders” by Levine — Ch. 3-4 |
| DWARF format | “The DWARF Debugging Standard” — dwarfstd.org |
| Stack unwinding | “Building a Debugger” by Sy Brand — Part 8-9 |
| Signal handling | “The Linux Programming Interface” — Ch. 20-22 |
| Multi-threading | “The Linux Programming Interface” — Ch. 29 + man ptrace (PTRACE_O_TRACECLONE) |
Quick Start Guide
For the overwhelmed learner, here’s your first 48 hours:
- Hour 1-2: Read Eli Bendersky’s “How debuggers work: Part 1” (blog)
- Hour 3-4: Run the minimal tracer from Project 1
- Hour 5-8: Complete Project 1 (fork/exec with TRACEME)
- Hour 9-12: Complete Project 2 (waitpid and signal handling)
- Hour 13-20: Complete Project 3-4 (read memory, read registers)
- Hour 21-30: Complete Project 5 (set breakpoints)
- Hour 31-40: Complete Project 6 (single-stepping)
- Hour 41-48: Complete Project 7 (continue with breakpoint handling)
After 48 hours: You have a minimal debugger that can attach, set breakpoints, step, and continue.
Recommended Learning Paths
Path A: Minimal Viable Debugger (8 weeks)
Focus on core functionality first.
Week 1-2: Projects 1-4 (ptrace basics, attach, memory, registers)
Week 3-4: Projects 5-7 (breakpoints, stepping, continue)
Week 5-6: Project 8 (signal handling) + Project 10 (ELF symbols)
Week 7-8: Project 11 (line tables) + basic UI improvements
Outcome: A debugger that can break on functions/lines, step, continue, print variables.
Path B: Full-Featured Debugger (16 weeks)
Complete all projects in order.
Weeks 1-4: Projects 1-8 (core debugging mechanics)
Weeks 5-8: Projects 9-12 (ELF/DWARF, source-level debugging)
Weeks 9-12: Projects 13-15 (watchpoints, expressions, stack)
Weeks 13-16: Project 16 (multi-threading) + polish
Outcome: A mini-GDB with most essential features.
Path C: Reverse Engineering Focus (10 weeks)
Emphasize binary analysis over source debugging.
Week 1-2: Projects 1-4 (ptrace basics)
Week 3-4: Projects 5-7 (breakpoints, stepping)
Week 5-6: Project 9 (ELF deep dive) + disassembly
Week 7-8: Project 13 (watchpoints) + memory analysis
Week 9-10: Custom: syscall tracing, code injection
Outcome: A tool focused on binary analysis rather than source-level debugging.
Project List
The projects build upon each other. Complete them in order for best results.
Project 1: Hello Tracer - Fork, Exec, and PTRACE_TRACEME
What you’ll build: A minimal program that launches a child process and traces it using ptrace. The parent will be notified when the child stops and can inspect its state.
Why it teaches the concept: This is the foundation of all debugging. Before you can set breakpoints or read memory, you must establish the tracer-tracee relationship.
Core challenges you’ll face:
- Understanding fork/exec semantics → The child must call
PTRACE_TRACEMEbeforeexec - Handling the initial stop → The tracee stops immediately after exec
- Synchronizing parent and child → Using
wait()correctly - Error handling → ptrace returns -1 on error with errno
Key Concepts:
- fork()/exec() mechanics: “The Linux Programming Interface” Ch. 24-27
- ptrace TRACEME:
man 2 ptrace - wait() status codes: “The Linux Programming Interface” Ch. 26
Difficulty: Beginner Time estimate: 4-6 hours Prerequisites: Basic C, familiarity with fork/exec
Real World Outcome
You will create a program called minidb that launches and traces a target program.
File structure:
minidb/
├── Makefile
├── src/
│ ├── main.c
│ └── tracer.c
└── test/
└── hello.c
Expected terminal session:
$ make
gcc -g -O0 -Wall -c src/main.c -o obj/main.o
gcc -g -O0 -Wall -c src/tracer.c -o obj/tracer.o
gcc -g -O0 obj/main.o obj/tracer.o -o minidb
$ ./minidb ./test/hello
[minidb] Starting target: ./test/hello
[minidb] Forked child with PID: 12345
[minidb] Child stopped with signal: SIGTRAP
[minidb] Child is ready for debugging
[minidb] Continuing child...
Hello, World!
[minidb] Child exited with status: 0
The tracer-tracee handshake:
Parent (tracer) Child (tracee)
│ │
│ fork() ───────────────────►│
│ │
│ ptrace(TRACEME)
│ │
│ execve("./target")
│ │
│◄── wait() ─────────────────│ SIGTRAP (stops)
│ │
│ ptrace(CONT) ─────────────►│ (continues)
│ │
│◄── wait() ─────────────────│ exit
│ │
The Core Question You’re Answering
How does a debugger establish control over another process at the operating system level?
Understanding this relationship is fundamental: ptrace creates a parent-child control channel that the kernel enforces. Every subsequent debugging operation depends on this foundation.
Concepts You Must Understand First
- What does
fork()return to the parent vs. the child?- Parent gets child’s PID (positive number)
- Child gets 0
- Reference: TLPI Ch. 24
- What happens when
execve()is called?- Process image is replaced
- PID stays the same
- Memory is reset but ptrace status is preserved
- Reference: TLPI Ch. 27
- What does
wait()return and how do you decode the status?- Returns PID of stopped/exited child
- Use
WIFEXITED,WIFSTOPPED,WSTOPSIGmacros - Reference: TLPI Ch. 26
Questions to Guide Your Design
Architecture:
- Should the tracer and tracee logic be in separate functions?
- How will you handle errors from system calls?
- What information should you print for debugging your debugger?
Implementation:
- In which process do you call
PTRACE_TRACEME- parent or child? - What system call do you use to wait for the child to stop?
- How do you know if the child exited vs. stopped by a signal?
Thinking Exercise
Before writing code, trace through this sequence on paper:
pid_t pid = fork();
if (pid == 0) {
// Child
ptrace(PTRACE_TRACEME, 0, NULL, NULL);
execl("./hello", "hello", NULL);
} else {
// Parent
int status;
waitpid(pid, &status, 0);
// What is status here?
}
- Draw the process tree after fork()
- What signal does the child receive after execve()?
- What value will
WSTOPSIG(status)return? - What happens if you remove the PTRACE_TRACEME call?
The Interview Questions They’ll Ask
-
Q: “What’s the difference between PTRACE_TRACEME and PTRACE_ATTACH?” A: TRACEME is called by the child to volunteer for tracing by its parent. ATTACH is called by the tracer to forcibly attach to an existing process (requires appropriate permissions).
-
Q: “Why does the child stop after execve() when traced?” A: The kernel sends SIGTRAP to a traced process after exec succeeds. This gives the tracer a chance to set breakpoints before any code runs.
-
Q: “Can any process trace any other process?” A: No. The tracer must have CAP_SYS_PTRACE capability or the target must be a child that called TRACEME. The Yama LSM can further restrict ptrace scope.
-
Q: “What happens to child processes of the tracee?” A: By default, they are not traced. You can use PTRACE_O_TRACEFORK, PTRACE_O_TRACECLONE to auto-trace children.
-
Q: “How does GDB handle the case where the target calls fork()?” A: GDB can follow the parent, follow the child, or detach from the child. This is controlled by
set follow-fork-mode.
Hints in Layers
Hint 1 (Conceptual Direction): The child must call ptrace before exec. The parent must wait for the child to stop.
Hint 2 (Approach): Use fork() to create the child. In the child, call ptrace(PTRACE_TRACEME, 0, NULL, NULL) then execl(). In the parent, call waitpid() and check if the child stopped.
Hint 3 (Technical Details):
// Child process
if (pid == 0) {
if (ptrace(PTRACE_TRACEME, 0, NULL, NULL) < 0) {
perror("ptrace TRACEME");
exit(1);
}
execl(target, target, NULL);
perror("execl");
exit(1);
}
Hint 4 (Verification):
- Use
strace -f ./minidb ./helloto see all system calls - Verify child receives SIGTRAP:
WSTOPSIG(status) == SIGTRAP - Check if child is stopped:
WIFSTOPPED(status)
Books That Will Help
| Topic | Book & Chapter |
|---|---|
| Process creation | “The Linux Programming Interface” Ch. 24-27 |
| ptrace basics | man 2 ptrace |
| Wait and status | “Advanced Programming in the UNIX Environment” Ch. 8 |
| Error handling | “The Linux Programming Interface” Ch. 3 |
Common Pitfalls & Debugging
| Problem | Cause | Fix |
|---|---|---|
| “Operation not permitted” | TRACEME called in parent | Call TRACEME in child before exec |
| Child runs without stopping | Missing TRACEME | Ensure TRACEME succeeds before exec |
| Parent hangs in wait | Child crashed before exec | Check execl return value |
| Wrong status value | Using wait() incorrectly | Use waitpid(pid, &status, 0) |
Quick verification test:
# Compile test target
echo 'int main() { return 42; }' > /tmp/test.c
gcc -o /tmp/test /tmp/test.c
# Run your tracer
./minidb /tmp/test
# Expected output includes:
# - "Child stopped with signal: SIGTRAP" or similar
# - "Child exited with status: 42" or similar
Learning Milestones
Milestone 1: Child process is created and receives TRACEME
- Verify with strace that ptrace(PTRACE_TRACEME) is called
Milestone 2: Parent receives notification when child stops
- waitpid returns and WIFSTOPPED(status) is true
- WSTOPSIG(status) returns SIGTRAP
Milestone 3: Parent can continue the child to completion
- ptrace(PTRACE_CONT) succeeds
- Child runs to completion and parent sees exit status
Project 2: Wait and See - Signal Handling and Wait Status
What you’ll build: Extend your tracer to properly decode all wait statuses (stopped, exited, signaled, continued) and handle multiple stops.
Why it teaches the concept: Real debugging involves handling many different events: breakpoint hits, signals, process exit, exec, and fork. You must decode wait status correctly to know what happened.
Core challenges you’ll face:
- Decoding wait status macros → WIFSTOPPED, WIFEXITED, WIFSIGNALED, etc.
- Distinguishing signal types → Differentiating SIGTRAP from other signals
- Handling ptrace events → PTRACE_O_TRACEEXEC, PTRACE_O_TRACEEXIT
- Creating an event loop → Repeatedly waiting for and handling events
Difficulty: Beginner Time estimate: 4-6 hours Prerequisites: Project 1 completed
Real World Outcome
Expected terminal session:
$ ./minidb ./test/signals
[minidb] Target PID: 23456
[minidb] Event: STOPPED by signal SIGTRAP (5)
[minidb] Continuing...
[minidb] Event: STOPPED by signal SIGSEGV (11)
[minidb] Signal: Segmentation fault at PC=0x401234
[minidb] Delivering signal to child...
[minidb] Event: SIGNALED by signal SIGSEGV (11)
[minidb] Child terminated by signal
Event loop structure:
while (1) {
int status;
pid_t pid = waitpid(child_pid, &status, 0);
if (WIFEXITED(status)) {
printf("Exited with code %d\n", WEXITSTATUS(status));
break;
} else if (WIFSIGNALED(status)) {
printf("Killed by signal %d\n", WTERMSIG(status));
break;
} else if (WIFSTOPPED(status)) {
int sig = WSTOPSIG(status);
printf("Stopped by signal %d\n", sig);
// Handle the stop, then continue
ptrace(PTRACE_CONT, child_pid, NULL, sig);
}
}
The Core Question You’re Answering
How do you distinguish between different reasons a traced process stops, and how do you correctly resume it?
The wait status is a packed integer with multiple meanings. Misinterpreting it means your debugger will misbehave.
Questions to Guide Your Design
- How do you know if the child exited normally vs. was killed by a signal?
- When the child stops on a signal, should you deliver that signal when continuing?
- How do you distinguish a breakpoint (SIGTRAP) from a real SIGTRAP the program sent itself?
The Interview Questions They’ll Ask
-
Q: “A user’s program handles SIGINT. How does your debugger interact with this?” A: The debugger sees the SIGINT as a stop event. It can choose to deliver the signal (pass it to the program) or suppress it. GDB’s “handle SIGINT nostop” changes this behavior.
-
Q: “What’s the difference between WSTOPSIG and WTERMSIG?” A: WSTOPSIG is the signal that caused a stop (from WIFSTOPPED). WTERMSIG is the signal that killed the process (from WIFSIGNALED).
-
Q: “How do ptrace-stop events differ from signal-delivery-stops?” A: Ptrace-stops (like breakpoints) use SIGTRAP with event info in the high bits. Signal-delivery-stops are regular signals. You check
(status >> 8) & 0xfffor additional event information.
Common Pitfalls & Debugging
| Problem | Cause | Fix |
|---|---|---|
| Signals not delivered | Passing 0 to PTRACE_CONT | Pass the signal number to deliver it |
| Extra stops | Not handling ptrace events | Enable PTRACE_O_TRACESYSGOOD |
| Infinite loop | Signal keeps firing | Suppress or handle recurring signals |
Project 3: Memory Inspector - Reading Process Memory
What you’ll build: A tool that can read and display memory from the traced process, including code, data, and stack regions.
Why it teaches the concept: Memory inspection is core to debugging. You need to examine variables, data structures, and code to understand program state.
Core challenges you’ll face:
- PTRACE_PEEKDATA semantics → Returns one word at a time, error checking is tricky
- Efficient bulk reads → Using
/proc/<pid>/memfor better performance - Address validation → Checking if addresses are mapped via
/proc/<pid>/maps - Display formatting → Hex dumps, ASCII representation
Difficulty: Beginner-Intermediate Time estimate: 6-8 hours Prerequisites: Projects 1-2 completed
Real World Outcome
Expected commands and output:
(minidb) x/32x 0x7fffffffde00
0x7fffffffde00: 0x00000001 0x00000000 0x7fffffffdf48 0x00000000
0x7fffffffde10: 0x00000000 0x00000000 0x7fffffffdf60 0x00000000
0x7fffffffde20: 0x00401210 0x00000000 0x00401040 0x00000000
0x7fffffffde30: 0x00000001 0x00000000 0x7fffffffdf48 0x00000000
(minidb) x/s 0x402000
0x402000: "Hello, World!\n"
(minidb) info proc mappings
Start End Perm Offset Device Inode Path
0x00400000 0x00401000 r--p 00000000 08:01 123456 /home/user/hello
0x00401000 0x00402000 r-xp 00001000 08:01 123456 /home/user/hello
0x00402000 0x00403000 r--p 00002000 08:01 123456 /home/user/hello
0x7ffff7dc0000 0x7ffff7df0000 r--p 00000000 08:01 789012 /lib/x86_64-linux-gnu/libc.so.6
...
Memory reading implementation approaches:
Approach 1: PTRACE_PEEKDATA (word-at-a-time)
┌─────────────────────────────────────────────────────────────────────┐
│ for each word: │
│ errno = 0; │
│ word = ptrace(PTRACE_PEEKDATA, pid, addr, NULL); │
│ if (errno != 0) handle_error(); │
│ addr += sizeof(word); │
└─────────────────────────────────────────────────────────────────────┘
Approach 2: /proc/<pid>/mem (bulk read, faster)
┌─────────────────────────────────────────────────────────────────────┐
│ fd = open("/proc/<pid>/mem", O_RDONLY); │
│ lseek(fd, addr, SEEK_SET); │
│ read(fd, buffer, size); │
│ close(fd); │
└─────────────────────────────────────────────────────────────────────┘
The Core Question You’re Answering
How do you read the memory of another process, and what are the trade-offs between different approaches?
Concepts You Must Understand First
- Virtual address spaces - Each process has its own address space
- Memory regions - Code, data, heap, stack, shared libraries
- Page permissions - Read, write, execute flags
- Word alignment - PEEKDATA returns aligned words
The Interview Questions They’ll Ask
-
Q: “Why does PTRACE_PEEKDATA return data in the return value instead of through a buffer?” A: Historical API design. It predates modern interfaces. The return value is both data (on success) and error indicator (returns -1 on error, but -1 could be valid data, so check errno).
-
Q: “How do you check if an address is readable before trying to read it?” A: Parse
/proc/<pid>/mapsto get memory regions and their permissions. -
Q: “Can you read memory while the process is running?” A: The process must be stopped (SIGSTOP or at a ptrace-stop) for reads to be safe and consistent.
Hints in Layers
Hint 1: Start with PTRACE_PEEKDATA for simplicity.
Hint 2: Clear errno before calling ptrace, check errno after.
Hint 3:
long read_word(pid_t pid, unsigned long addr) {
errno = 0;
long word = ptrace(PTRACE_PEEKDATA, pid, addr, NULL);
if (errno != 0) {
perror("PEEKDATA");
return -1;
}
return word;
}
Hint 4: For bulk reads, open /proc/<pid>/mem with the child stopped.
Project 4: Register Dump - Reading and Writing CPU Registers
What you’ll build: Functionality to read and display all CPU registers, and modify specific registers.
Why it teaches the concept: Registers hold the current state of computation. RIP tells you where execution is, RSP tells you about the stack, and other registers hold function arguments and local values.
Core challenges you’ll face:
- Using PTRACE_GETREGS → Fills a
struct user_regs_struct - Understanding register layout → Different fields for different registers
- Setting registers → PTRACE_SETREGS with modified struct
- Register naming conventions → rax vs eax vs ax
Difficulty: Beginner-Intermediate Time estimate: 4-6 hours Prerequisites: Projects 1-3 completed
Real World Outcome
Expected output:
(minidb) info registers
rax 0x0 0
rbx 0x0 0
rcx 0x7ffff7ea1100 140737352700160
rdx 0x7fffffffde18 140737488346648
rsi 0x7fffffffde08 140737488346632
rdi 0x1 1
rbp 0x7fffffffdd10 0x7fffffffdd10
rsp 0x7fffffffdd10 0x7fffffffdd10
r8 0x0 0
r9 0x7ffff7fc9f60 140737353924448
r10 0x7ffff7fc3908 140737353898248
r11 0x206 518
r12 0x401040 4198464
r13 0x7fffffffde00 140737488346624
r14 0x0 0
r15 0x0 0
rip 0x401136 0x401136 <main+10>
eflags 0x246 [ PF ZF IF ]
cs 0x33 51
ss 0x2b 43
(minidb) set $rax = 0x42
(minidb) info registers rax
rax 0x42 66
Register structure on x86-64:
struct user_regs_struct {
unsigned long r15, r14, r13, r12;
unsigned long rbp, rbx;
unsigned long r11, r10, r9, r8;
unsigned long rax, rcx, rdx, rsi, rdi;
unsigned long orig_rax;
unsigned long rip, cs;
unsigned long eflags;
unsigned long rsp, ss;
unsigned long fs_base, gs_base;
unsigned long ds, es, fs, gs;
};
The Core Question You’re Answering
How do you inspect and modify the CPU state of a traced process, and what does each register mean?
The Interview Questions They’ll Ask
-
Q: “What is orig_rax and when is it useful?” A: orig_rax stores the system call number when stopped at a syscall entry. rax may be modified by the kernel, but orig_rax preserves the original value for syscall identification.
-
Q: “Why is RIP one past the INT3 instruction when hitting a breakpoint?” A: The CPU executes INT3 (which is 1 byte), then traps. RIP is incremented to the next instruction before the trap is processed. The debugger must decrement RIP to point back to the breakpoint location.
-
Q: “How would you get the floating-point registers?” A: Use PTRACE_GETFPREGS or PTRACE_GETREGSET with NT_PRFPREG for SSE/AVX state.
Project 5: Breakpoint Basics - Setting Software Breakpoints
What you’ll build: The ability to set breakpoints by address, replacing instructions with INT3 and restoring them.
Why it teaches the concept: Breakpoints are the core mechanism for stopping execution at specific points. Understanding how they work demystifies one of debugging’s most fundamental features.
Core challenges you’ll face:
- Saving original bytes → Must restore them to continue
- Handling multi-byte instructions → INT3 is 1 byte, but may overlay longer instructions
- Managing breakpoint state → Track enabled/disabled/hit status
- Handling the RIP adjustment → RIP points past INT3 when stopped
Difficulty: Intermediate Time estimate: 8-12 hours Prerequisites: Projects 1-4 completed
Real World Outcome
Expected session:
(minidb) break *0x401136
Breakpoint 1 at 0x401136
(minidb) run
Starting program: ./hello
Breakpoint 1, 0x401136 in main ()
(minidb) info breakpoints
Num Type Address Enabled Hit Count
1 breakpoint 0x0000000000401136 y 1
(minidb) delete 1
Deleted breakpoint 1
(minidb) continue
Hello, World!
[Inferior 1 exited normally]
Breakpoint data structure:
struct breakpoint {
unsigned long addr; // Address where breakpoint is set
uint8_t saved_byte; // Original byte at that address
bool enabled; // Is breakpoint active?
int hit_count; // How many times it was hit
};
Setting a breakpoint (pseudo-code):
1. Read the byte at target address:
saved_byte = PTRACE_PEEKDATA(addr) & 0xff
2. Write INT3 (0xCC) to the address:
word = PTRACE_PEEKDATA(addr)
word = (word & ~0xff) | 0xCC
PTRACE_POKEDATA(addr, word)
3. Store breakpoint info in our list
The Core Question You’re Answering
How do you make a program stop at a specific address, and how do you then resume execution correctly?
Concepts You Must Understand First
- INT3 opcode (0xCC) - Single-byte trap instruction
- Instruction boundaries - Must know where instructions start
- SIGTRAP handling - Breakpoint generates SIGTRAP
- RIP adjustment - CPU increments RIP past INT3
Questions to Guide Your Design
- How do you modify a single byte when POKEDATA writes a word (8 bytes)?
- What happens if you set a breakpoint in the middle of an instruction?
- How do you handle the case where the user sets a breakpoint at an address that already has one?
The Interview Questions They’ll Ask
-
Q: “What’s the difference between a software breakpoint and a hardware breakpoint?” A: Software breakpoints modify code (INT3). Hardware breakpoints use CPU debug registers (DR0-DR3). Hardware breakpoints don’t modify code and can break on data access, but are limited in number (usually 4).
-
Q: “How do you handle breakpoints in read-only code sections?” A: Use mprotect to make the page writable temporarily, write the INT3, then restore permissions. Or use hardware breakpoints.
-
Q: “How does GDB handle breakpoints in shared libraries that aren’t loaded yet?” A: GDB sets a pending breakpoint and intercepts the dynamic linker’s notification when libraries are loaded, then activates the breakpoint.
Common Pitfalls & Debugging
| Problem | Cause | Fix |
|---|---|---|
| “Invalid argument” on POKEDATA | Unaligned address | Read full word, modify byte, write back |
| Breakpoint never hits | Wrong address | Verify with objdump or readelf |
| Crash after breakpoint | Forgot to restore byte | Restore before single-step |
| Infinite breakpoint loop | Forgot to single-step past | Step past, then re-insert |
Learning Milestones
Milestone 1: Can set breakpoint and process stops at that address
- SIGTRAP received, RIP is at addr+1
Milestone 2: Can inspect state at breakpoint
- Read registers, read memory at breakpoint location
Milestone 3: Can continue past breakpoint
- Restore byte, decrement RIP, single-step, re-insert INT3, continue
Project 6: Single Step - Instruction-Level Execution
What you’ll build: The ability to execute one instruction at a time (single-stepping) and display the current instruction.
Why it teaches the concept: Single-stepping is essential for detailed debugging. Understanding how the CPU’s trap flag enables this reveals low-level CPU debugging support.
Core challenges you’ll face:
- PTRACE_SINGLESTEP mechanics → Sets CPU trap flag, executes one instruction
- Disassembly integration → Showing what instruction was executed
- Breakpoint interaction → Stepping over breakpoints correctly
- Branch handling → What happens when stepping over a call?
Difficulty: Intermediate Time estimate: 6-8 hours Prerequisites: Project 5 completed
Real World Outcome
Expected session:
(minidb) break main
Breakpoint 1 at 0x401136
(minidb) run
Breakpoint 1, 0x401136 in main ()
(minidb) stepi
0x401138 in main ()
(minidb) stepi
0x40113b in main ()
(minidb) x/i $rip
=> 0x40113b <main+15>: mov DWORD PTR [rbp-0x4],0x0
(minidb) stepi 5
0x40115a in main ()
How single-stepping works:
CPU Trap Flag (TF) in EFLAGS:
EFLAGS register bit 8 = Trap Flag
┌─────────────────────────────────────────────────────────────┐
│ Bit: 31...9 8 7 6 5 4 3 2 1 0 │
│ ... TF IF DF SF ZF AF PF CF │
└─────────────────────────────────────────────────────────────┘
When TF=1:
1. CPU executes exactly ONE instruction
2. CPU generates debug exception (#DB)
3. Kernel delivers SIGTRAP to tracee
4. Tracee stops, tracer notified via wait()
5. TF is automatically cleared
ptrace(PTRACE_SINGLESTEP, pid, NULL, NULL):
1. Sets TF=1 in tracee's EFLAGS
2. Continues tracee
3. Tracee executes one instruction
4. Tracee stops with SIGTRAP
The Core Question You’re Answering
How do you execute exactly one machine instruction and regain control?
The Interview Questions They’ll Ask
-
Q: “What’s the difference between
stepiandstepin GDB?” A:stepisteps one machine instruction.stepsteps one source line (which may be many instructions).stepalso steps into function calls, whilenextsteps over them. -
Q: “How do you implement ‘step over’ (stepping over a call instruction)?” A: Set a temporary breakpoint at the instruction after the call, then continue. When the breakpoint hits, remove it.
-
Q: “Can you single-step into a system call?” A: Not with SINGLESTEP. Use PTRACE_SYSCALL to stop at syscall entry and exit.
Common Pitfalls & Debugging
| Problem | Cause | Fix |
|---|---|---|
| Steps multiple instructions | Nested signals | Check for correct SIGTRAP handling |
| Hangs during step | Signal not handled | Ensure SINGLESTEP returns immediately |
| Breakpoint missed | Stepped past it | Handle breakpoints before stepping |
Project 7: Continue and Resume - Correct Breakpoint Handling
What you’ll build: Proper continue functionality that handles breakpoints correctly, including temporary disable, single-step, and re-enable.
Why it teaches the concept: The interaction between breakpoints and continue is tricky. You must handle the case where you’re stopped at a breakpoint and want to continue.
Core challenges you’ll face:
- The continue-at-breakpoint problem → Must execute the original instruction
- Breakpoint restoration → Re-insert after stepping past
- Multiple breakpoints → Track which one(s) are hit
- Conditional breakpoints → Continue automatically if condition false
Difficulty: Intermediate Time estimate: 8-10 hours Prerequisites: Projects 5-6 completed
Real World Outcome
Expected session with multiple breakpoints:
(minidb) break *0x401136
Breakpoint 1 at 0x401136
(minidb) break *0x401150
Breakpoint 2 at 0x401150
(minidb) run
Breakpoint 1, 0x401136 in main ()
(minidb) continue
Breakpoint 2, 0x401150 in main ()
(minidb) continue
Hello, World!
[Inferior exited normally]
The continue-at-breakpoint dance:
State: Stopped at breakpoint at 0x401136
RIP = 0x401137 (one past INT3)
Original instruction was: 48 89 e5 (mov rbp, rsp)
Step 1: Disable breakpoint
- Write back original byte: 48
- Memory now: 48 89 e5 ...
Step 2: Decrement RIP
- RIP = 0x401136
Step 3: Single-step
- PTRACE_SINGLESTEP
- CPU executes: mov rbp, rsp
- RIP = 0x401139
Step 4: Re-enable breakpoint
- Write INT3: CC
- Memory now: CC 89 e5 ...
Step 5: Continue
- PTRACE_CONT
- Process runs until next breakpoint or exit
The Core Question You’re Answering
How do you resume execution from a breakpoint without missing the original instruction or hitting the same breakpoint immediately?
The Interview Questions They’ll Ask
-
Q: “What happens if two breakpoints are at consecutive addresses?” A: After stepping past the first, you might immediately hit the second. The debugger must handle nested breakpoint hits.
-
Q: “How do you implement ‘continue until address’?” A: Set a temporary breakpoint at the target address, continue, then remove the temporary breakpoint when hit.
-
Q: “How do you handle the case where single-stepping hits another breakpoint?” A: Treat it as a new breakpoint hit. The original breakpoint will be re-enabled after handling the new one.
Project 8: Signal Mastery - Handling All Process Signals
What you’ll build: Complete signal handling including delivering signals to the target, intercepting signals, and proper signal pass-through.
Why it teaches the concept: Real programs use signals. Your debugger must correctly handle SIGINT, SIGSEGV, SIGFPE, and other signals the target might receive.
Core challenges you’ll face:
- Signal delivery decision → Suppress, deliver, or transform signals
- SIGSEGV handling → Help user debug crashes
- SIGINT handling → Ctrl+C should stop target, not debugger
- Signal chaining → What if signal handler causes another signal?
Difficulty: Intermediate Time estimate: 6-8 hours Prerequisites: Projects 1-7 completed
Real World Outcome
Expected session with signal handling:
(minidb) handle SIGINT stop print
Signal Stop Print Pass to program
SIGINT Yes Yes No
(minidb) run
^C
Program received signal SIGINT, Interrupt.
0x7ffff7ea1100 in read () from /lib/x86_64-linux-gnu/libc.so.6
(minidb) continue
Continuing with signal SIGINT.
(program handles SIGINT or terminates)
(minidb) handle SIGSEGV stop print nopass
(minidb) run
Program received signal SIGSEGV, Segmentation fault.
0x401150 in main ()
(minidb) # Signal not delivered, we can inspect
The Core Question You’re Answering
How do you intercept, inspect, modify, or suppress signals destined for the debugged program?
The Interview Questions They’ll Ask
-
Q: “How do you tell the difference between a SIGTRAP from a breakpoint and a SIGTRAP from a kill(pid, SIGTRAP)?” A: Check the signal info using PTRACE_GETSIGINFO. The si_code field tells you the source (SI_KERNEL for breakpoint, SI_USER for kill).
-
Q: “How do you pass a signal to the target when continuing?” A: The fourth argument to PTRACE_CONT is the signal number to deliver (or 0 for no signal).
-
Q: “What happens if the target installs a signal handler for SIGTRAP?” A: The debugger sees the stop first. If it delivers the signal, the target’s handler runs. Debuggers typically suppress SIGTRAP from breakpoints to avoid confusing the target.
Project 9: ELF Explorer - Parsing the Binary Format
What you’ll build: An ELF parser that extracts symbol tables, section headers, and program headers from the target binary.
Why it teaches the concept: Source-level debugging requires mapping between function names, source files, and addresses. ELF symbols provide the function name to address mapping.
Core challenges you’ll face:
- ELF header parsing → Magic number, class, endianness
- Section headers → Finding .symtab, .strtab, .dynsym
- Symbol table interpretation → Name, address, size, type
- Dynamic symbols → Shared library functions
Difficulty: Intermediate-Advanced Time estimate: 10-12 hours Prerequisites: Projects 1-8 completed
Real World Outcome
Expected output:
(minidb) info file
ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
Class: ELF64
Data: 2's complement, little endian
Type: EXEC (Executable file)
Machine: Advanced Micro Devices X86-64
Entry point address: 0x401040
Section Headers:
[Nr] Name Type Address Size
[ 1] .text PROGBITS 0000000000401000 00001a2
[ 2] .rodata PROGBITS 0000000000402000 0000015
...
(minidb) info functions
0x0000000000401040 _start
0x0000000000401070 __libc_csu_init
0x00000000004010e0 __libc_csu_fini
0x0000000000401126 main
0x0000000000401180 helper_func
(minidb) break main
Breakpoint 1 at 0x401126
ELF parsing flow:
┌──────────────────┐
│ ELF File │
└────────┬─────────┘
│
┌────────▼─────────┐
│ ELF Header │
│ (e_shoff, e_phoff)│
└────────┬─────────┘
│
┌─────────────────┴─────────────────┐
│ │
┌────────▼─────────┐ ┌──────────▼──────────┐
│ Program Headers │ │ Section Headers │
│ (for loading) │ │ (for linking/debug)│
└──────────────────┘ └──────────┬──────────┘
│
┌────────────────┼────────────────┐
│ │ │
┌─────────▼───┐ ┌───────▼────┐ ┌───────▼────┐
│ .symtab │ │ .strtab │ │ .debug_* │
│ (symbols) │ │ (strings) │ │ (DWARF) │
└─────────────┘ └────────────┘ └────────────┘
The Core Question You’re Answering
How do you extract symbol information from an ELF binary to map function names to addresses?
Concepts You Must Understand First
- ELF file structure - Headers, sections, segments
- Symbol tables - Name index, address, size, type, binding
- String tables - Null-terminated strings indexed by offset
- Section vs segment - Sections for linker, segments for loader
The Interview Questions They’ll Ask
-
Q: “What’s the difference between .symtab and .dynsym?” A: .symtab contains all symbols including local ones (may be stripped). .dynsym contains only dynamic symbols needed for runtime linking (never stripped for dynamic executables).
-
Q: “How do you find the address of a function by name?” A: Parse .symtab or .dynsym, for each symbol entry, look up the name in .strtab using st_name offset, compare with target name, return st_value if matched.
-
Q: “Can you set a breakpoint in a stripped binary?” A: Not by function name (no symbols). You can still break by address, or use DWARF info if debug sections remain.
Common Pitfalls & Debugging
| Problem | Cause | Fix |
|---|---|---|
| No symbols found | Binary is stripped | Check .dynsym or rebuild with symbols |
| Wrong endianness | Not checking EI_DATA | Read e_ident[EI_DATA] |
| Crash parsing | Wrong offset calculations | Use e_shentsize, e_shnum |
Project 10: DWARF Decoder - Basic Debug Information
What you’ll build: A DWARF parser that can read compilation units, extract function information, and map addresses to source locations.
Why it teaches the concept: DWARF is the bridge between machine code and source code. Understanding it is essential for source-level debugging.
Core challenges you’ll face:
- DWARF format complexity → DIEs, attributes, forms, abbreviations
- .debug_info parsing → Compilation units, tags, attributes
- .debug_abbrev interpretation → Abbreviation tables
- Address to DIE mapping → Finding which DIE covers an address
Difficulty: Advanced Time estimate: 15-20 hours Prerequisites: Project 9 completed
Real World Outcome
Expected output:
(minidb) info sources
Source files:
main.c
helper.c
(minidb) info types
Types defined in main.c:
struct Point { int x; int y; }
typedef struct Point Point;
(minidb) ptype Point
type = struct Point {
int x;
int y;
}
(minidb) whatis argc
type = int
DWARF DIE structure example:
<0><b>: Abbrev Number: 1 (DW_TAG_compile_unit)
DW_AT_producer : GNU C17 11.3.0
DW_AT_language : 12 (ANSI C)
DW_AT_name : main.c
DW_AT_comp_dir : /home/user/project
DW_AT_low_pc : 0x401126
DW_AT_high_pc : 0x4011a0
DW_AT_stmt_list : 0x0
<1><50>: Abbrev Number: 2 (DW_TAG_subprogram)
DW_AT_name : main
DW_AT_decl_file : 1
DW_AT_decl_line : 5
DW_AT_type : <0x80>
DW_AT_low_pc : 0x401126
DW_AT_high_pc : 0x4011a0
DW_AT_frame_base : ...
<2><75>: Abbrev Number: 3 (DW_TAG_formal_parameter)
DW_AT_name : argc
DW_AT_decl_file : 1
DW_AT_decl_line : 5
DW_AT_type : <0x80>
DW_AT_location : DW_OP_fbreg -20
The Core Question You’re Answering
How do you decode DWARF debug information to understand the relationship between addresses, source files, functions, and variables?
The Interview Questions They’ll Ask
- Q: “What are the main .debug_* sections and what do they contain?”
A:
- .debug_info: DIE tree (types, functions, variables)
- .debug_abbrev: Abbreviation tables for decoding info
- .debug_line: Line number information
- .debug_frame: Call frame information for unwinding
- .debug_str: String table
-
Q: “What is a DIE (Debugging Information Entry)?” A: A DIE is a node in the DWARF tree representing an entity like a function, variable, type, or compilation unit. It has a tag (DW_TAG_) and attributes (DW_AT_).
- Q: “How do you find the source line for a given address?” A: Parse .debug_line, build the line table state machine, find the row that contains the address.
Project 11: Line Number Tables - Source to Address Mapping
What you’ll build: A line table parser that maps source file:line to addresses and vice versa.
Why it teaches the concept: Line tables are essential for break file.c:42 and for showing the current source line during stepping.
Core challenges you’ll face:
- Line table state machine → DWARF uses a state machine encoding
- Special opcodes → Compact encoding for common cases
- Extended opcodes → End sequence, set address, etc.
- Building the lookup table → Efficient address <-> line mapping
Difficulty: Advanced Time estimate: 12-15 hours Prerequisites: Project 10 completed
Real World Outcome
Expected session:
(minidb) break main.c:10
Breakpoint 1 at 0x401136: file main.c, line 10.
(minidb) run
Breakpoint 1, main (argc=1, argv=0x7fffffffde08) at main.c:10
10 int x = 42;
(minidb) list
5 int main(int argc, char **argv) {
6 printf("Arguments: %d\n", argc);
7
8 for (int i = 0; i < argc; i++) {
9 printf(" %s\n", argv[i]);
10 }
11
12 int x = 42; <-- current line
13 int y = x * 2;
14
15 return 0;
16 }
(minidb) next
13 int y = x * 2;
Line table state machine:
Line Table State Machine:
Initial state:
address = 0
file = 1
line = 1
column = 0
is_stmt = default_is_stmt
basic_block = false
end_sequence = false
prologue_end = false
epilogue_begin = false
Opcodes modify state and emit rows:
┌──────────────────────────────────────────────────────────────────┐
│ Standard Opcodes: │
│ DW_LNS_copy - emit row, reset flags │
│ DW_LNS_advance_pc - advance address │
│ DW_LNS_advance_line - advance line │
│ DW_LNS_set_file - change current file │
│ DW_LNS_set_column - set column │
│ DW_LNS_negate_stmt - toggle is_stmt │
│ DW_LNS_const_add_pc - advance address by constant │
│ │
│ Special Opcodes (computed): │
│ address += (opcode - opcode_base) / line_range │
│ line += line_base + (opcode - opcode_base) % line_range │
│ emit row │
│ │
│ Extended Opcodes: │
│ DW_LNE_end_sequence - end of sequence marker │
│ DW_LNE_set_address - set address directly │
│ DW_LNE_define_file - define new file │
└──────────────────────────────────────────────────────────────────┘
The Core Question You’re Answering
How do you convert between source file/line numbers and machine code addresses?
The Interview Questions They’ll Ask
-
Q: “Why does DWARF use a state machine for line tables instead of a simple table?” A: Compression. Source-to-address mapping is repetitive (consecutive lines often map to consecutive addresses). The state machine encodes deltas compactly.
-
Q: “What does ‘is_stmt’ mean in the line table?” A: ‘is_stmt’ indicates if this is a “statement” - a recommended breakpoint location. Helps debuggers find meaningful stop points.
-
Q: “How do you handle inlined functions in line tables?” A: The line table maps addresses to the original source location. Inlined code shows the original source. To also show the call site, you need .debug_info’s DW_TAG_inlined_subroutine entries.
Project 12: Source Display - Showing Code Context
What you’ll build: Integration of line tables with source file reading to display source code during debugging.
Why it teaches the concept: A debugger is only useful if users can see what code they’re debugging. This connects all previous work.
Core challenges you’ll face:
- Finding source files → Using compilation directory and file paths
- Caching source content → Don’t re-read files constantly
- Handling missing sources → What if source isn’t available?
- Syntax highlighting → Making output readable (optional)
Difficulty: Intermediate Time estimate: 8-10 hours Prerequisites: Projects 10-11 completed
Real World Outcome
Expected session:
(minidb) list main
1 #include <stdio.h>
2
3 void helper(int n) {
4 printf("Helper: %d\n", n);
5 }
6
7 int main(int argc, char **argv) {
8 printf("Arguments: %d\n", argc);
9 helper(argc);
10 return 0;
11 }
(minidb) break 8
Breakpoint 1 at 0x401136: file main.c, line 8.
(minidb) run
Breakpoint 1, main () at main.c:8
8 printf("Arguments: %d\n", argc);
(minidb) next
9 helper(argc);
The Core Question You’re Answering
How do you display the relevant source code at the current execution point?
Project 13: Watchpoints - Hardware Data Breakpoints
What you’ll build: Watchpoints that stop execution when a memory location is read or written, using hardware debug registers.
Why it teaches the concept: Watchpoints are invaluable for finding “who changed this variable?” bugs. Understanding debug registers reveals CPU debugging support.
Core challenges you’ll face:
- Debug register setup → DR0-DR3 for addresses, DR7 for control
- Watch types → Read, write, read/write, execute
- Size limitations → 1, 2, 4, or 8 bytes
- Maximum watchpoints → Only 4 hardware watchpoints on x86
Difficulty: Advanced Time estimate: 10-12 hours Prerequisites: Projects 1-8 completed
Real World Outcome
Expected session:
(minidb) print &x
$1 = (int *) 0x7fffffffdd0c
(minidb) watch *0x7fffffffdd0c
Hardware watchpoint 1: *0x7fffffffdd0c
(minidb) continue
Hardware watchpoint 1: *0x7fffffffdd0c
Old value = 42
New value = 100
0x401155 in main () at main.c:12
12 x = 100;
(minidb) info watchpoints
Num Type Address Value
1 hw watchpoint 0x7fffffffdd0c 100
(minidb) continue
Hello, World!
[Inferior exited normally]
Watchpoint 1 deleted because the program has left the block
Debug register layout (x86-64):
DR7 (Debug Control Register):
┌─────────────────────────────────────────────────────────────────────┐
│ Bit │ Field │ Description │
├───────┼───────┼────────────────────────────────────────────────────┤
│ 0 │ L0 │ Local enable for DR0 │
│ 1 │ G0 │ Global enable for DR0 │
│ 2 │ L1 │ Local enable for DR1 │
│ 3 │ G1 │ Global enable for DR1 │
│ 4 │ L2 │ Local enable for DR2 │
│ 5 │ G2 │ Global enable for DR2 │
│ 6 │ L3 │ Local enable for DR3 │
│ 7 │ G3 │ Global enable for DR3 │
│ 16-17 │ RW0 │ Condition for DR0 (00=exec,01=write,11=rw) │
│ 18-19 │ LEN0 │ Length for DR0 (00=1,01=2,10=8,11=4 bytes) │
│ 20-21 │ RW1 │ Condition for DR1 │
│ 22-23 │ LEN1 │ Length for DR1 │
│ 24-25 │ RW2 │ Condition for DR2 │
│ 26-27 │ LEN2 │ Length for DR2 │
│ 28-29 │ RW3 │ Condition for DR3 │
│ 30-31 │ LEN3 │ Length for DR3 │
└───────┴───────┴────────────────────────────────────────────────────┘
The Core Question You’re Answering
How do you detect when a specific memory location is accessed without modifying the code?
The Interview Questions They’ll Ask
-
Q: “Why are hardware watchpoints limited to 4 on x86?” A: The CPU provides 4 debug address registers (DR0-DR3). This is a hardware limitation. Software watchpoints have no limit but are much slower.
-
Q: “How do software watchpoints work as a fallback?” A: Single-step the program, checking the watched address after each instruction. Extremely slow but unlimited.
-
Q: “What happens if the watched variable goes out of scope?” A: The watchpoint should be deleted or disabled. GDB tracks scopes using DWARF and deletes watchpoints when their scope ends.
Project 14: Expression Evaluator - Printing Variables and Expressions
What you’ll build: An expression parser and evaluator that can handle C expressions like print x->next->data + 5.
Why it teaches the concept: Simply reading memory isn’t enough - you need to understand types and navigate data structures.
Core challenges you’ll face:
- Expression parsing → Operator precedence, member access, dereference
- Type lookup → Find variable types in DWARF
- Location expressions → DWARF location descriptions
- Type-aware memory reading → Integers, floats, pointers, structs
Difficulty: Advanced Time estimate: 15-20 hours Prerequisites: Projects 10-12 completed
Real World Outcome
Expected session:
(minidb) print argc
$1 = 3
(minidb) print argv[0]
$2 = 0x7fffffffdf48 "/home/user/hello"
(minidb) print sizeof(struct Point)
$3 = 8
(minidb) print node->next->value
$4 = 42
(minidb) print (int)(x * 1.5)
$5 = 63
(minidb) set x = 100
(minidb) print x
$6 = 100
Expression evaluation flow:
User input: "node->next->value"
│
▼
┌───────────────────┐
│ Lexer/Parser │ Tokenize and parse
└─────────┬─────────┘
│
▼
┌───────────────────┐
│ AST: │
│ Member("value") │
│ │ │
│ Member("next") │
│ │ │
│ Ident("node") │
└─────────┬─────────┘
│
▼
┌───────────────────┐
│ Symbol Lookup │ Find "node" in DWARF
│ Type: Node* │ Address from location expr
│ Addr: rbp-8 │
└─────────┬─────────┘
│
▼
┌───────────────────┐
│ Read Memory │ value = PEEK(rbp-8)
│ Type: Node* │ = 0x555555558000
└─────────┬─────────┘
│
▼
┌───────────────────┐
│ Member "next" │ Type: Node*, offset 8
│ Read: 0x555...+8 │ = 0x555555558020
└─────────┬─────────┘
│
▼
┌───────────────────┐
│ Member "value" │ Type: int, offset 0
│ Read: 0x555...+0 │ = 42
└─────────┬─────────┘
│
▼
Result: 42
The Core Question You’re Answering
How do you evaluate C expressions in the context of a stopped program, using debug information to understand types and locations?
The Interview Questions They’ll Ask
-
Q: “How do you find where a local variable is stored?” A: Look up the variable’s DIE in DWARF. The DW_AT_location attribute contains a DWARF expression that computes the address (often relative to frame base or a register).
-
Q: “What’s a DWARF location expression?” A: A stack-based bytecode that computes an address. Operations include push constant, add register value, dereference, and arithmetic.
-
Q: “How do you handle optimized-out variables?” A: The location expression might not cover the current PC, or the DIE might have DW_AT_location_list pointing to pieces. Report “value optimized out” if no valid location.
Project 15: Stack Unwinder - Walking the Call Stack
What you’ll build: A stack unwinder that produces backtraces with function names, arguments, and source locations.
Why it teaches the concept: The backtrace is essential for understanding how you got here. This requires understanding both stack layout and DWARF CFI.
Core challenges you’ll face:
- Frame pointer unwinding → Follow RBP chain when available
- DWARF CFI unwinding → Use .debug_frame when no frame pointers
- Showing arguments → Read parameters using DWARF info
- Handling signal frames → Special handling for signal trampolines
Difficulty: Advanced Time estimate: 12-15 hours Prerequisites: Projects 10-12 completed
Real World Outcome
Expected session:
(minidb) backtrace
#0 0x00007ffff7ea1100 in read () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x0000000000401234 in get_input (buffer=0x7fffffffdc00, size=256) at io.c:42
#2 0x0000000000401180 in process_command () at main.c:28
#3 0x0000000000401156 in main (argc=1, argv=0x7fffffffde08) at main.c:15
(minidb) frame 2
#2 0x0000000000401180 in process_command () at main.c:28
28 char *cmd = get_input(buffer, sizeof(buffer));
(minidb) info locals
buffer = {0x7fffffffdc00 "help\n", ...}
cmd = 0x0
(minidb) info args
(no arguments)
(minidb) up
#3 0x0000000000401156 in main (argc=1, argv=0x7fffffffde08) at main.c:15
(minidb) info args
argc = 1
argv = 0x7fffffffde08
Stack unwinding approaches:
Method 1: Frame Pointer Chain (when -fno-omit-frame-pointer)
┌────────────────────────────────────────────────────────────────────┐
│ Start: RBP of current frame │
│ │
│ Loop: │
│ saved_rbp = *(RBP) # Previous frame's RBP │
│ return_addr = *(RBP + 8) # Where this function returns │
│ RBP = saved_rbp # Move to previous frame │
│ │
│ Stop when: RBP == 0 or RBP points outside stack │
└────────────────────────────────────────────────────────────────────┘
Method 2: DWARF CFI (Call Frame Information)
┌────────────────────────────────────────────────────────────────────┐
│ Start: Current RIP, RSP │
│ │
│ Loop: │
│ Find FDE covering current RIP │
│ Apply CFI rules to compute: │
│ CFA = RSP + offset (or other rule) │
│ return_addr = *(CFA - 8) │
│ caller's RSP = CFA │
│ RIP = return_addr │
│ │
│ Stop when: No FDE found or at _start │
└────────────────────────────────────────────────────────────────────┘
The Core Question You’re Answering
How do you reconstruct the call stack to show how the program reached the current point?
The Interview Questions They’ll Ask
-
Q: “Why would unwinding fail?” A: Corrupted stack, missing CFI data (stripped binary, JIT code), stack pointer corruption, leaf functions without frame setup.
-
Q: “What’s the CFA (Canonical Frame Address)?” A: The CFA is the stack pointer value just before the call instruction. All other frame data is expressed relative to CFA.
-
Q: “How do you unwind through a signal handler?” A: The kernel saves the interrupted state in a signal frame on the stack. DWARF has special CIE augmentations (signal_frame flag) to handle this.
Project 16: Thread Support - Multi-Threaded Debugging Basics
What you’ll build: Basic support for debugging multi-threaded programs: listing threads, switching threads, and handling thread events.
Why it teaches the concept: Modern programs are multi-threaded. Debugging threads requires understanding how ptrace handles multiple tasks.
Core challenges you’ll face:
- Thread discovery → Using /proc/
/task/ or PTRACE_O_TRACECLONE - Thread switching → Each thread has its own register state
- All-stop vs. non-stop → Stop all threads or just one?
- Thread-specific breakpoints → Breaking only in certain threads
Difficulty: Advanced Time estimate: 15-20 hours Prerequisites: All previous projects
Real World Outcome
Expected session:
(minidb) run
[Thread 1] Created new thread 2
[Thread 1] Created new thread 3
(minidb) info threads
Id Target Id Frame
* 1 Thread 0x7ffff7fc9700 (LWP 12345) "main" main () at main.c:25
2 Thread 0x7ffff7fc8700 (LWP 12346) "worker" worker_func () at worker.c:10
3 Thread 0x7ffff7fc7700 (LWP 12347) "worker" worker_func () at worker.c:10
(minidb) thread 2
[Switching to thread 2 (Thread 0x7ffff7fc8700 (LWP 12346))]
#0 worker_func (arg=0x1) at worker.c:10
10 while (!done) {
(minidb) bt
#0 worker_func (arg=0x1) at worker.c:10
#1 0x00007ffff7f9b6db in start_thread (arg=0x7ffff7fc8700) from /lib/x86_64-linux-gnu/libpthread.so.0
#2 0x00007ffff7ec261f in clone () from /lib/x86_64-linux-gnu/libc.so.6
(minidb) break worker.c:15 thread 2
Breakpoint 1 at 0x401234: file worker.c, line 15.
Thread handling with ptrace:
Thread Events via PTRACE_O_TRACECLONE:
1. Enable clone tracking:
ptrace(PTRACE_SETOPTIONS, pid, 0, PTRACE_O_TRACECLONE);
2. When tracee creates thread:
- New thread starts in stopped state
- Parent receives PTRACE_EVENT_CLONE
- Get new thread's TID: ptrace(PTRACE_GETEVENTMSG, pid, 0, &new_tid);
3. Managing multiple threads:
- Each thread has its own /proc/<tid>/
- Each thread can be stopped/continued independently
- Register state is per-thread
4. All-stop mode (default):
- When one thread stops, stop all threads
- Continue resumes all threads
5. Non-stop mode (advanced):
- Threads run independently
- Only stopped threads are controlled
The Core Question You’re Answering
How do you debug programs with multiple threads, handling thread creation, switching between threads, and setting thread-specific breakpoints?
The Interview Questions They’ll Ask
-
Q: “How do you list all threads of a process?” A: Read
/proc/<pid>/task/directory. Each subdirectory is a thread ID. Or use PTRACE_O_TRACECLONE to track thread creation. -
Q: “What’s the difference between all-stop and non-stop mode?” A: In all-stop, when one thread stops (breakpoint, signal), all threads stop. In non-stop, only the affected thread stops; others continue running.
-
Q: “How do you handle thread-specific breakpoints?” A: Set the breakpoint normally, but when hit, check which thread hit it. If not the target thread, continue without stopping.
Project Comparison Table
| # | Project | Difficulty | Time | Core Skill |
|---|---|---|---|---|
| 1 | Hello Tracer | Beginner | 4-6h | fork/exec/ptrace basics |
| 2 | Wait and See | Beginner | 4-6h | Signal handling, wait status |
| 3 | Memory Inspector | Beginner-Int | 6-8h | PEEK/POKE, /proc/mem |
| 4 | Register Dump | Beginner-Int | 4-6h | GETREGS/SETREGS |
| 5 | Breakpoint Basics | Intermediate | 8-12h | INT3, instruction patching |
| 6 | Single Step | Intermediate | 6-8h | SINGLESTEP, trap flag |
| 7 | Continue and Resume | Intermediate | 8-10h | Breakpoint dance |
| 8 | Signal Mastery | Intermediate | 6-8h | Signal delivery/suppression |
| 9 | ELF Explorer | Int-Advanced | 10-12h | ELF parsing, symbols |
| 10 | DWARF Decoder | Advanced | 15-20h | DIEs, attributes, types |
| 11 | Line Number Tables | Advanced | 12-15h | Source mapping |
| 12 | Source Display | Intermediate | 8-10h | File handling, integration |
| 13 | Watchpoints | Advanced | 10-12h | Debug registers |
| 14 | Expression Evaluator | Advanced | 15-20h | Parsing, type system |
| 15 | Stack Unwinder | Advanced | 12-15h | CFI, frame walking |
| 16 | Thread Support | Advanced | 15-20h | Multi-threaded debugging |
Summary
| Project | What You Build | Key Learning |
|---|---|---|
| 1-2 | Tracer foundation | ptrace API, process control |
| 3-4 | Memory/register access | Inspecting process state |
| 5-7 | Breakpoints + stepping | Core debugging mechanics |
| 8 | Signal handling | Real-world signal interaction |
| 9-12 | ELF/DWARF/source | Symbol and source mapping |
| 13-14 | Watchpoints + expressions | Advanced data inspection |
| 15-16 | Stack + threads | Call tracking, concurrency |
After completing all projects, you will have:
- A functional mini-GDB that can debug C programs
- Deep understanding of how debuggers interact with the OS
- Knowledge of ELF and DWARF binary formats
- Skills applicable to reverse engineering, profiling, and tracing tools
References and Further Reading
Primary Resources
- “Building a Debugger” by Sy Brand - Comprehensive tutorial series
- https://blog.tartanllama.xyz/writing-a-linux-debugger-setup/
- “How debuggers work” by Eli Bendersky - Three-part series
- Part 1: https://eli.thegreenplace.net/2011/01/23/how-debuggers-work-part-1
- Part 2: https://eli.thegreenplace.net/2011/01/27/how-debuggers-work-part-2-breakpoints
- Part 3: https://eli.thegreenplace.net/2011/02/07/how-debuggers-work-part-3-debugging-information
- “The Art of Debugging with GDB, DDD, and Eclipse” by Norman Matloff and Peter Jay Salzman
- Practical debugging techniques and tool usage
- ptrace(2) man page
man 2 ptrace- Complete system call reference
Binary Format References
- ELF Specification
- https://refspecs.linuxfoundation.org/elf/elf.pdf
- DWARF Debugging Standard
- https://dwarfstd.org/doc/DWARF5.pdf
- “Linkers and Loaders” by John R. Levine
- ELF format and linking details
System Programming
- “The Linux Programming Interface” by Michael Kerrisk
- Ch. 24-27: Process creation and signals
- Ch. 44: Pipes and FIFOs (for debugger-debuggee communication)
- “Computer Systems: A Programmer’s Perspective” (CS:APP) by Bryant and O’Hallaron
- Ch. 3: Machine-level representation
- Ch. 8: Exceptional control flow
- Intel 64 and IA-32 Architectures Software Developer’s Manual
- Volume 3: Chapter 17 - Debug, Branch Profile, TSC, and Resource Monitoring
Related Projects and Source Code
- GDB Source Code: https://sourceware.org/git/binutils-gdb.git
- LLDB Source Code: https://github.com/llvm/llvm-project/tree/main/lldb
- rr (record and replay debugger): https://rr-project.org/
- strace source: https://github.com/strace/strace
Final Notes
Building a debugger is one of the most educational systems programming projects you can undertake. You will:
- Understand how your tools work at the deepest level
- Gain skills that transfer to reverse engineering, profiling, and tracing
- Learn to read kernel documentation and binary format specifications
- Build confidence in low-level programming
Remember: GDB took decades to develop. Your mini-debugger won’t match its features, but the knowledge you gain will be invaluable. Start simple, iterate often, and don’t be afraid to study the source code of real debuggers.
Good luck, and happy debugging!
Last updated: 2025