Project 7: Linux GDB Watchpoints for Game State

Use GDB or LLDB watchpoints to stop execution exactly when HP changes on Linux.

Quick Reference

Attribute Value
Difficulty Level 3
Time Estimate 1-2 weeks
Main Programming Language Tool-driven (GDB/LLDB)
Alternative Programming Languages N/A
Coolness Level Level 4
Business Potential Level 1
Prerequisites P02 scan workflow
Key Topics Watchpoints, stack traces

1. Learning Objectives

By completing this project, you will:

  1. Set a watchpoint on a target address in Linux.
  2. Capture the instruction and call stack responsible for HP changes.
  3. Document the Linux workflow equivalent of Project 4.

2. All Theory Needed (Per-Concept Breakdown)

Watchpoints and Call Stack Analysis

  • Fundamentals Watchpoints stop execution when a memory address is accessed or modified. They are the most precise way to learn which instruction updates a value. Combined with a call stack, watchpoints let you trace the high-level path that caused the update.

  • Deep Dive into the concept Watchpoints are implemented using hardware debug registers on most CPUs. This makes them extremely precise but also limited: only a small number of watchpoints can be active at once. This limitation affects strategy: you must choose the most important addresses to watch. In a lab, that is usually the authoritative HP value discovered through scanning or prior analysis.

    When a watchpoint triggers, GDB (or LLDB) stops the process and shows the current instruction. The call stack reveals the chain of functions that led to the write. This is the Linux equivalent of control-flow recon in OllyDbg. Because Linux binaries often use PIE and ASLR by default, addresses can change between runs, so you may need to set watchpoints using symbol names or compute addresses relative to a module base.

    Another key concept is that watchpoints can be triggered by many writes. If the value is updated frequently, the watchpoint will fire too often, and you may lose the signal. The solution is to engineer a controlled scenario where the value changes at a predictable moment (using the deterministic sandbox). This keeps the signal-to-noise ratio high.

    Call stack analysis is the method of turning the watchpoint trigger into a narrative. You label each frame with its role: for example, game_loop calls apply_damage, which calls update_health. By doing this multiple times, you build confidence in the flow. When you can explain why the write happened and which decision led to it, you have effectively reverse engineered that behavior.

  • How this fit on projects This concept is the Linux counterpart to Project 4 and prepares you for Project 8 and the Capstone.

  • Definitions & key terms
    • Watchpoint: Stop on memory access or write.
    • Backtrace: Stack trace of active calls.
    • PIE: Position-independent executable.
  • Mental model diagram ``` Watchpoint Trigger

[HP Address] <- write -> [Instruction] <- [Call Stack]


- **How it works (step-by-step)**
  1. Identify HP address (from scanning or symbols).
  2. Set a watchpoint on the address.
  3. Trigger a hit event.
  4. Record instruction and backtrace.

- **Minimal concrete example**

Watchpoint hit: HP 100 -> 95 #0 update_health #1 apply_damage #2 game_loop


- **Common misconceptions**
  - “Watchpoints are unlimited.” (They are not.)
  - “The first frame is always the root cause.” (Callers matter.)

- **Check-your-understanding questions**
  1. Why are hardware watchpoints limited?
  2. How does PIE affect address stability?
  3. Why is determinism important for watchpoints?

- **Check-your-understanding answers**
  1. CPUs have limited debug registers.
  2. PIE randomizes base addresses, shifting runtime addresses.
  3. It reduces noise by making the trigger predictable.

- **Real-world applications**
  - Debugging subtle state changes in complex systems.

- **Where you’ll apply it**
  - This project’s §5.4 and §6.2.
  - Also used in: P08-linux-proc-ptrace-observer.md, P10-cross-platform-game-hacking-lab.md.

- **References**
  - GDB documentation (watchpoints).
  - Practical Reverse Engineering - Ch. 1.

- **Key insights**
  Watchpoints give you the precise moment of change and the call path that caused it.

- **Summary**
  Watchpoints plus stack analysis are the Linux equivalent of Windows control-flow recon.

- **Homework/Exercises to practice the concept**
  1. Identify one address that changes per tick and watch it.
  2. Compare the call stack for two different events.

- **Solutions to the homework/exercises**
  1. Choose a value with predictable changes (HP or timer).
  2. Note which function differs between events.

---

## 3. Project Specification

### 3.1 What You Will Build

A Linux debugging workflow that uses watchpoints to catch HP changes, captures backtraces, and documents the execution path for damage events.

### 3.2 Functional Requirements

1. Set a watchpoint on HP.
2. Trigger a hit event.
3. Record instruction and backtrace.
4. Repeat across at least two runs.

### 3.3 Non-Functional Requirements

- **Performance**: Watchpoint triggers within one hit event.
- **Reliability**: Same call path appears across runs.
- **Usability**: Documentation clear and reproducible.

### 3.4 Example Usage / Output

Watchpoint hit: HP 100 -> 95 #0 update_health #1 apply_damage #2 game_loop


### 3.5 Data Formats / Schemas / Protocols

- **Backtrace log**: event -> instruction -> stack frames.

### 3.6 Edge Cases

- Value changes too frequently.
- Wrong address watched.

### 3.7 Real World Outcome

#### 3.7.1 How to Run (Copy/Paste)

1. Launch sandbox under GDB.
2. Set a write watchpoint on HP.
3. Trigger the hit event.

#### 3.7.2 Golden Path Demo (Deterministic)

Watchpoint triggers exactly once and the stack is captured.

#### 3.7.3 Failure Demo

Watchpoint triggers repeatedly because HP changes every tick.

---

## 4. Solution Architecture

### 4.1 High-Level Design

[HP Address] -> [Watchpoint] -> [Instruction + Backtrace] -> [Documentation]


### 4.2 Key Components

| Component | Responsibility | Key Decisions |
|-----------|----------------|---------------|
| Watchpoint | Stop on write | Correct address |
| Backtrace | Capture call chain | Which frames to keep |

### 4.3 Data Structures (No Full Code)

- **Backtrace log** table with frames and notes.

### 4.4 Algorithm Overview

**Key Algorithm: Watchpoint Trigger**
1. Set watchpoint.
2. Trigger event.
3. Record stack.

**Complexity Analysis**
- Time: O(1) per trigger.
- Space: O(depth of stack).

---

## 5. Implementation Guide

### 5.1 Development Environment Setup

- Linux VM with GDB/LLDB.

### 5.2 Project Structure

project/ ├── backtrace-notes.md └── screenshots/


### 5.3 The Core Question You're Answering

> "Which instruction changes HP on Linux, and what path leads to it?"

### 5.4 Concepts You Must Understand First

- Watchpoints
- Call stacks

### 5.5 Questions to Guide Your Design

1. How will you identify the correct HP address?
2. How will you reduce noise from frequent updates?

### 5.6 Thinking Exercise

Draw a call stack with three frames and label each one.

### 5.7 The Interview Questions They'll Ask

1. "Why are watchpoints limited?"
2. "How do you interpret a backtrace?"
3. "What is PIE?"

### 5.8 Hints in Layers

**Hint 1:** Use deterministic sandbox to reduce noise.
**Hint 2:** Verify address via freeze.
**Hint 3:** Use backtrace to label frames.
**Hint 4:** Repeat across runs.

### 5.9 Books That Will Help

| Topic | Book | Chapter |
|-------|------|---------|
| Debugging | Art of Debugging with GDB | Ch. 2-3 |

### 5.10 Implementation Phases

#### Phase 1: Address Identification (2 hours)
- Locate HP address via scan or symbols.

**Checkpoint:** Address confirmed.

#### Phase 2: Watchpoint Setup (2 hours)
- Set watchpoint and trigger event.

**Checkpoint:** Watchpoint triggers.

#### Phase 3: Documentation (4 hours)
- Record backtrace and annotate.

**Checkpoint:** Call chain documented.

### 5.11 Key Implementation Decisions

| Decision | Options | Recommendation | Rationale |
|----------|---------|----------------|-----------|
| Address source | scan vs symbols | scan | Consistent with Windows workflow |

---

## 6. Testing Strategy

### 6.1 Test Categories

| Category | Purpose | Examples |
|----------|---------|----------|
| Repeatability | Same stack | Run twice |
| Accuracy | Correct instruction | Compare logs |

### 6.2 Critical Test Cases

1. Same instruction hit across runs.
2. Stack frames match expected functions.

### 6.3 Test Data

Expected: update_health <- apply_damage <- game_loop ```

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

Pitfall Symptom Solution
Wrong address Watchpoint never hits Re-validate value
Too noisy Too many stops Use deterministic scenario

7.2 Debugging Strategies

  • Narrow to a single state change.
  • Use conditional watchpoints if available.

8. Extensions & Challenges

8.1 Beginner Extensions

  • Watch a timer variable instead of HP.

8.2 Intermediate Extensions

  • Compare watchpoints for HP and ammo.

8.3 Advanced Extensions

  • Identify and annotate branch conditions leading to HP change.

9. Real-World Connections

9.1 Industry Applications

  • Debugging memory corruption and state anomalies.
  • Sandbox from P01.

9.3 Interview Relevance

  • Watchpoints and debugging are common systems questions.

10. Resources

10.1 Essential Reading

  • Art of Debugging with GDB - Ch. 2-3

10.2 Video Resources

  • GDB watchpoint demonstrations.

10.3 Tools & Documentation

  • GDB and LLDB manuals.
  • P04-ollydbg-control-flow-recon.md
  • P08-linux-proc-ptrace-observer.md

11. Self-Assessment Checklist

11.1 Understanding

  • I can explain how watchpoints work.
  • I can interpret a backtrace.

11.2 Implementation

  • Watchpoint triggers reliably.
  • Backtrace documented.

11.3 Growth

  • I can reproduce the workflow on a new variable.

12. Submission / Completion Criteria

Minimum Viable Completion:

  • Watchpoint triggers and stack recorded.

Full Completion:

  • Stack annotations and cross-run validation.

Excellence (Going Above & Beyond):

  • Comparative analysis of multiple value changes.