Project 3: The Memory Inspector

Build a program with a controlled buffer overflow to corrupt stack data, then use GDB’s memory examination commands on the core dump to become a crash detective and understand why the program failed, not just where.

Quick Reference

Attribute	Value
Difficulty	Intermediate
Time Estimate	1-2 weeks
Language	C (alt: C++)
Prerequisites	Project 2 (GDB Backtrace), basic understanding of pointers and arrays
Key Topics	Stack layout, buffer overflows, GDB memory examination (x command), variable inspection, stack frame navigation

1. Learning Objectives

By completing this project, you will:

Understand stack memory layout: Know exactly how local variables, return addresses, and function parameters are arranged on the stack
Write controlled buffer overflows: Create reproducible memory corruption scenarios that demonstrate security vulnerabilities
Master GDB memory examination: Use the x (examine) command fluently with different formats, sizes, and counts
Inspect corrupted variables: Use print (p) to see how buffer overflows change variable values
Navigate stack frames: Switch between function contexts using frame to inspect variables at different call levels
Connect cause to effect: Trace memory corruption from its source (the overflow) to its consequence (the crash)
Think like a detective: Develop the mental model of treating a crash dump as a “crime scene” to investigate

2. Theoretical Foundation

2.1 Core Concepts

The Stack: Your Function’s Workspace

Every time a function is called, it gets a “stack frame” - a contiguous block of memory on the stack that holds:

High Memory Addresses
┌─────────────────────────────────────────┐
│         Previous Stack Frame            │
├─────────────────────────────────────────┤
│     Return Address (pushed by call)     │  ← Where to go when function returns
├─────────────────────────────────────────┤
│         Saved Base Pointer (RBP)        │  ← Previous frame's base
├─────────────────────────────────────────┤  ← Current RBP points here
│         Local Variable 1                │
├─────────────────────────────────────────┤
│         Local Variable 2                │
├─────────────────────────────────────────┤
│         Local Array/Buffer              │  ← If overflow happens here...
│         [element 0]                     │
│         [element 1]                     │
│         ...                             │
│         [element n]                     │  ← ...it grows UPWARD into other vars!
├─────────────────────────────────────────┤  ← Current RSP points here
│         (Future stack growth)           │
└─────────────────────────────────────────┘
Low Memory Addresses

Critical insight: On x86/x86-64, the stack grows DOWNWARD (toward lower addresses), but when you write past the end of an array, you overflow UPWARD (toward higher addresses). This means buffer overflows can corrupt:

Other local variables
Saved base pointer (RBP)
Return address
Caller’s stack frame

Buffer Overflow Mechanics

A buffer overflow occurs when data is written beyond the bounds of an allocated buffer:

void vulnerable_function(char *input) {
    char local_buffer[64];  // Only 64 bytes allocated
    strcpy(local_buffer, input);  // No bounds checking!
    // If input > 64 bytes, we overflow into adjacent stack memory
}

Before Overflow:               After Overflow (200 'A's):
┌─────────────────┐            ┌─────────────────┐
│ Return Address  │            │ 0x41414141...   │ ← CORRUPTED!
├─────────────────┤            ├─────────────────┤
│ Saved RBP       │            │ 0x41414141...   │ ← CORRUPTED!
├─────────────────┤            ├─────────────────┤
│ other_var = 42  │            │ 0x41414141 (A)  │ ← CORRUPTED!
├─────────────────┤            ├─────────────────┤
│ local_buffer[63]│            │ 'A' (0x41)      │
│ local_buffer[62]│            │ 'A' (0x41)      │
│ ...             │            │ ...             │
│ local_buffer[0] │            │ 'A' (0x41)      │
└─────────────────┘            └─────────────────┘

GDB Memory Examination Commands

The x (examine) command is your primary tool for viewing raw memory:

x/[count][format][size] address

count:  How many units to display
format: How to interpret the data
        x = hexadecimal (most common)
        d = signed decimal
        u = unsigned decimal
        o = octal
        t = binary
        c = character
        s = string
        i = instruction (disassembly)
size:   Unit size
        b = byte (1 byte)
        h = halfword (2 bytes)
        w = word (4 bytes)
        g = giant (8 bytes, useful for 64-bit)

Essential command patterns:

x/16wx $rsp          # 16 4-byte words at stack pointer (hex)
x/8gx $rbp           # 8 8-byte giants at base pointer (64-bit addresses)
x/32xb &buffer       # 32 bytes at buffer address (byte-by-byte)
x/s &string_var      # Display as null-terminated string
x/10i $rip           # Disassemble 10 instructions from current position

2.2 Why This Matters

Understanding memory examination in crash analysis is crucial because:

Backtraces lie (sometimes): A backtrace shows you WHERE the crash happened, but memory corruption may have occurred long before. The real bug could be in a completely different function.
Security implications: Buffer overflows are the foundation of many exploits. Understanding them defensively helps you write secure code and recognize vulnerabilities in code review.
Complex debugging: In multi-threaded programs or programs with extensive state, the backtrace alone doesn’t tell you enough. Memory inspection reveals the actual state of your data.
Performance debugging: Sometimes “corruption” isn’t malicious - it’s a use-after-free, double-free, or uninitialized memory. Memory inspection helps distinguish these cases.

2.3 Historical Context

Buffer overflows have been a security concern since at least the 1988 Morris Worm, which exploited a buffer overflow in the fingerd daemon. The technique was popularized by Aleph One’s 1996 article “Smashing the Stack for Fun and Profit” in Phrack Magazine.

Modern systems include multiple mitigations:

Stack canaries: Random values placed before return addresses; if overwritten, program terminates
ASLR (Address Space Layout Randomization): Randomizes memory addresses
NX/DEP (Non-Executable stack): Prevents executing code on the stack
Position-Independent Executables (PIE): Code addresses are randomized

For learning purposes, we’ll disable some protections to see the vulnerability clearly. In production, always enable all available protections.

2.4 Common Misconceptions

Misconception 1: “Buffer overflows always cause immediate crashes” Reality: Not necessarily. The overflow might corrupt data that isn’t used until much later, causing a delayed or intermittent crash. This makes root cause analysis essential.

Misconception 2: “The print command in GDB shows me everything I need” Reality: print shows you the interpreted value of a variable according to its type. examine (x) shows you the raw bytes, which is essential when data has been corrupted or when debugging type punning issues.

Misconception 3: “The crash location is where the bug is” Reality: Memory corruption often causes crashes far from the original bug. The actual overflow might happen in function A, but the crash occurs in function B when it tries to use corrupted data.

Misconception 4: “Arrays overflow in the direction they’re indexed” Reality: On x86/x86-64, arrays on the stack overflow toward HIGHER addresses (upward in memory), which happens to be toward the function’s caller. This is backwards from what many people expect.

3. Project Specification

3.1 What You Will Build

A C program called corrupter that:

Declares a local variable (secret_code) with a known value
Declares a buffer that will be overflowed
Calls a function that performs an unsafe copy operation (deliberate buffer overflow)
Crashes in a way that can be analyzed

You will then perform post-mortem analysis on the core dump to:

Identify what crashed
Inspect the corrupted variable
Examine raw stack memory to see the overflow pattern
Determine the exact cause of the corruption

3.2 Functional Requirements

The Vulnerable Program:
- Must have at least two functions (main and a vulnerable function)
- Must contain a variable that gets corrupted by the overflow
- Must use an unsafe string operation (strcpy or memcpy without bounds checking)
- Must crash in a reproducible way
The Overflow Data:
- Use a recognizable pattern (e.g., all ‘A’s = 0x41) so it’s visible in memory dumps
- Overflow must be large enough to corrupt adjacent variables
- Overflow should be controllable (you specify how much to overflow)
The Analysis Process:
- Load core dump into GDB
- Get backtrace to identify crash location
- Switch to appropriate frame to inspect variables
- Use print to see corrupted values
- Use examine to see raw memory and the overflow pattern

3.3 Non-Functional Requirements

Compile with debug symbols (-g) to enable meaningful variable inspection
Disable stack protector (-fno-stack-protector) to allow the overflow to succeed (for learning)
Enable core dumps (ulimit -c unlimited)
Reproducible: Same input produces same corruption pattern
Educational: Code should be clear and well-commented to explain what’s happening

3.4 Example Usage / Output

Running the vulnerable program:

$ ulimit -c unlimited
$ gcc -g -fno-stack-protector -o corrupter corrupter.c
$ ./corrupter
Segmentation fault (core dumped)

Analyzing the core dump:

$ gdb ./corrupter core.12345
GNU gdb (GDB) 12.1
...
Core was generated by './corrupter'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000555555555169 in vulnerable_function (input=0x7fffffffdc60 'A' <repeats 200 times>) at corrupter.c:15

(gdb) bt
#0  0x0000555555555169 in vulnerable_function (input=0x7fffffffdc60 'A' <repeats 200 times>) at corrupter.c:15
#1  0x00005555555551d8 in main () at corrupter.c:26

(gdb) frame 1
#1  0x00005555555551d8 in main () at corrupter.c:26

(gdb) p secret_code
$1 = 1094795585  # Should be 1337, but it's now 0x41414141!

(gdb) p/x secret_code
$2 = 0x41414141  # 'AAAA' - the overflow pattern!

(gdb) p buffer
$3 = "AAAAAAAAAAAAAAAAAAAAAAAAAA..."

(gdb) x/32xb &secret_code
0x7fffffffdca0: 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x41
0x7fffffffdca8: 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x41
...
# The entire region has been overwritten with 0x41 ('A')!

3.5 Real World Outcome

When complete, you will be able to:

Create the crashing program:

$ ./corrupter
Starting program...
secret_code is: 1337
Calling vulnerable_function with 200 bytes of 'A'...
Segmentation fault (core dumped)

Load and analyze in GDB: ```gdb $ gdb ./corrupter core.1234 (gdb) bt #0 0x0000555555555169 in vulnerable_function (…) at corrupter.c:15 #1 0x00005555555551d8 in main () at corrupter.c:26

Let’s see what main’s secret_code looks like

(gdb) frame 1 #1 0x00005555555551d8 in main () at corrupter.c:26

(gdb) info locals secret_code = 1094795585 buffer = “AAAAAAAAAA…”

That’s not 1337! Let’s see it in hex:

(gdb) p/x secret_code $1 = 0x41414141

0x41 is ASCII ‘A’ - the buffer overflow pattern!

Let’s examine raw memory around secret_code:

(gdb) x/8wx &secret_code 0x7fffffffdc90: 0x41414141 0x41414141 0x41414141 0x41414141 0x7fffffffdca0: 0x41414141 0x41414141 0x41414141 0x41414141

The entire region is filled with ‘A’s (0x41)

Now let’s look at the stack to see the full picture:

(gdb) x/32wx $rbp-128 0x7fffffffdc10: 0x41414141 0x41414141 0x41414141 0x41414141 0x7fffffffdc20: 0x41414141 0x41414141 0x41414141 0x41414141 …

We can see exactly where the overflow started and how far it went


3. **Understand the vulnerability**:

Memory Layout in main(): ┌──────────────────┐ │ Return Address │ ← Possibly corrupted too! ├──────────────────┤ │ Saved RBP │ Higher addresses ├──────────────────┤ ↑ │ secret_code │ ← CORRUPTED: now 0x41414141 │ │ (4 bytes) │ │ ├──────────────────┤ Overflow goes │ │ this way │ buffer[127] │ ← Overflow continues here │ │ buffer[126] │ │ │ … │ │ │ buffer[64] │ ← Original buffer ends here │ … │ │ buffer[1] │ │ buffer[0] │ ← Overflow starts copying here Lower addresses └──────────────────┘

---

## 4. Solution Architecture

### 4.1 High-Level Design

┌─────────────────────────────────────────────────────────────────────────────┐ │ corrupter.c │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ main() │ │ ┌─────────────────────────────────────────────────────────────────────────┐│ │ │ • Declare secret_code = 1337 ││ │ │ • Declare buffer[128] (will be source of overflow data) ││ │ │ • Fill buffer with attack pattern (‘A’ * 200) ││ │ │ • Call vulnerable_function(buffer) ││ │ │ • Program may or may not reach this point (depends on overflow size) ││ │ └─────────────────────────────────────────────────────────────────────────┘│ │ │ │ │ ▼ │ │ vulnerable_function(char *input) │ │ ┌─────────────────────────────────────────────────────────────────────────┐│ │ │ • Declare local_buffer[64] (smaller than input!) ││ │ │ • strcpy(local_buffer, input) ← OVERFLOW HAPPENS HERE ││ │ │ • … crash when return address or other data is corrupted ││ │ └─────────────────────────────────────────────────────────────────────────┘│ │ │ └─────────────────────────────────────────────────────────────────────────────┘ │ ▼ SIGSEGV (crash) │ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ │ CORE DUMP FILE │ │ Contains: All memory at crash time, register values, stack state │ └─────────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ │ GDB ANALYSIS SESSION │ │ 1. bt (backtrace) - see call chain │ │ 2. frame N - switch to frame N │ │ 3. p variable - print variable value │ │ 4. x/NFS addr - examine N units of size S in format F at addr │ └─────────────────────────────────────────────────────────────────────────────┘

### 4.2 Key Components

| Component | Purpose | Key Details |
|-----------|---------|-------------|
| **main()** | Setup and trigger | Initializes secret value, prepares attack data, calls vulnerable function |
| **vulnerable_function()** | Contains the vulnerability | Has small local buffer, uses unsafe strcpy |
| **Attack pattern** | Recognizable data | Using 'A' (0x41) makes overflow visible in memory dumps |
| **Debug symbols** | Enable inspection | `-g` flag ensures variable names are available |
| **Disabled protections** | Allow learning | `-fno-stack-protector` lets overflow succeed |

### 4.3 Data Structures

The key "data structures" here are the stack frames themselves:

```c
// main()'s stack frame (conceptual)
struct main_stack_frame {
    // Return address (inserted by call instruction)
    // Saved RBP
    int secret_code;          // 4 bytes, value = 1337
    char buffer[128];         // Attack payload goes here
    // possibly padding for alignment
};

// vulnerable_function()'s stack frame (conceptual)
struct vulnerable_stack_frame {
    // Return address
    // Saved RBP
    char local_buffer[64];    // Too small! Overflow will corrupt above
    // possibly padding
};

4.4 Algorithm Overview

Overflow Algorithm:

1. main() stores secret_code = 1337 on its stack
2. main() fills buffer with 200 'A' characters
3. main() calls vulnerable_function(buffer)
4. vulnerable_function() allocates local_buffer[64] on stack
5. strcpy(local_buffer, buffer) copies 200 bytes into 64-byte space
6. Overflow corrupts: other local vars, saved RBP, return address
7. When function returns or uses corrupted data: CRASH

Stack at step 6:
High addr ─┬─ [return address] ← CORRUPTED: 0x41414141...
           │  [saved RBP]      ← CORRUPTED: 0x41414141...
           │  [local_buffer[63]] = 'A'
           │  [local_buffer[62]] = 'A'
           │  ...
           │  [local_buffer[0]]  = 'A'
Low addr  ─┴─ RSP points here

Analysis Algorithm:

Load: gdb ./corrupter core.XXXX
Locate: bt (backtrace shows crash location)
Context: frame 1 (switch to main's frame)
Inspect: p secret_code (see corrupted value)
Format: p/x secret_code (see as hex - recognize 0x41414141)
Examine: x/32xb &secret_code (see raw bytes)
Correlate: Recognize 0x41 pattern = our overflow data
Conclude: Buffer overflow from vulnerable_function corrupted secret_code

5. Implementation Guide

5.1 Development Environment Setup

# Install required tools (Ubuntu/Debian)
sudo apt-get update
sudo apt-get install build-essential gdb

# Create project directory
mkdir -p memory-inspector
cd memory-inspector

# Enable core dumps for this session
ulimit -c unlimited

# Check core dump settings
cat /proc/sys/kernel/core_pattern
# If it shows a complex pattern like "|/usr/share/apport/apport %p...",
# you may want to simplify for learning:
# sudo sysctl -w kernel.core_pattern=core.%p
# (This makes core files appear in current directory as core.PID)

# Verify GDB is working
gdb --version

5.2 Project Structure

memory-inspector/
├── corrupter.c          # Main vulnerable program
├── Makefile             # Build configuration
├── analyze.sh           # Script to run analysis
├── README.md            # Project documentation
└── artifacts/           # Generated files (optional)
    ├── core.*           # Core dump files
    └── analysis.txt     # Analysis notes

Makefile:

CC = gcc
CFLAGS = -g -fno-stack-protector -z execstack
TARGET = corrupter

all: $(TARGET)

$(TARGET): corrupter.c
	$(CC) $(CFLAGS) -o $@ $<

clean:
	rm -f $(TARGET) core.*

run: $(TARGET)
	ulimit -c unlimited && ./$(TARGET)

.PHONY: all clean run

5.3 The Core Question You’re Answering

“How can I use GDB to understand not just WHERE a crash happened, but WHY it happened by examining the actual state of memory at the time of the crash?”

The backtrace tells you the call chain leading to the crash. But memory examination tells you the story - what data was in memory, how it got corrupted, and which operation corrupted it. This is the difference between knowing “the car crashed” and understanding “the brakes failed because the brake line was cut.”

5.4 Concepts You Must Understand First

Before starting this project, verify you understand these concepts:

Concept	Self-Assessment Question	Where to Learn
Stack layout	What order are local variables stored on the stack? Which direction does it grow?	CS:APP Ch. 3.7, “Dive Into Systems” Ch. 6
Pointers and arrays	What’s the relationship between `buffer` and `&buffer[0]`?	K&R Ch. 5
String operations	Why is `strcpy` dangerous? What does it NOT check?	Any C book’s string chapter
Hexadecimal	What is 0x41 in decimal? In ASCII?	CS:APP Ch. 2.1
GDB basics	How do you set a breakpoint? Print a variable?	Project 2 of this series
Core dumps	What is a core dump? When is it created?	Project 1 of this series

5.5 Questions to Guide Your Design

Work through these questions BEFORE writing code:

Stack Organization:
- If you declare int a; char buffer[100]; int b; in a function, what’s the relative position of a and b in memory? Which one could buffer overflow affect?
- Why does the order of variable declaration sometimes not match the order in memory?
Overflow Size:
- If your target buffer is 64 bytes, how much overflow do you need to corrupt a variable that’s (conceptually) above it on the stack?
- What factors affect this? (hint: alignment, compiler decisions)
Pattern Choice:
- Why use a recognizable pattern like ‘A’ (0x41)? What would happen if you used random data?
- What other patterns might be useful? (hint: incrementing values, address-like patterns)
Detection:
- How will you know the crash is due to corruption vs. the overflow itself?
- What would the crash look like if the return address was corrupted vs. just a local variable?
GDB Workflow:
- Which stack frame will show you secret_code? The frame where the crash happened or its caller?
- How do you know when to use print vs. examine?

5.6 Thinking Exercise

Before writing any code, trace through this scenario by hand:

void copy_data(char *src) {
    char dest[8];
    strcpy(dest, src);  // src is "AAAAAAAAAAAAAAAA" (16 bytes)
}

int main() {
    int x = 42;
    char data[20] = "AAAAAAAAAAAAAAAA";
    copy_data(data);
    return x;  // What value is returned?
}

Questions to answer on paper:

Draw the stack layout when copy_data is called. Show:
- main’s stack frame (x, data, return address)
- copy_data’s stack frame (dest, return address to main)
When strcpy(dest, src) executes with 16 bytes (15 chars + null):
- How many bytes overflow past dest[7]?
- What memory locations are corrupted?
What are the possible outcomes when copy_data tries to return?
- If only padding is corrupted?
- If the saved RBP is corrupted?
- If the return address is corrupted?
In GDB, if you’re at the crash and run frame 1 then p x, what might you see?

5.7 Hints in Layers

Reveal hints one at a time only if stuck:

Hint 1: Basic Structure

Start with this skeleton:

#include <stdio.h>
#include <string.h>

void vulnerable_function(char *input) {
    char local_buffer[64];
    // TODO: unsafe copy here
}

int main() {
    int secret_code = 1337;
    char buffer[256];

    printf("secret_code is: %d\n", secret_code);

    // TODO: fill buffer with attack pattern
    // TODO: call vulnerable_function

    printf("secret_code is now: %d\n", secret_code);
    return 0;
}

Compile with: gcc -g -fno-stack-protector -o corrupter corrupter.c

Hint 2: Creating the Attack Pattern

Use memset to fill your buffer with a recognizable pattern:

memset(buffer, 'A', 200);  // Fill 200 bytes with 'A' (0x41)
buffer[199] = '\0';        // Null terminate for strcpy

Why 200? It should be larger than local_buffer (64) plus enough to overflow into other variables. The exact amount depends on stack alignment and compiler decisions.

Hint 3: The Vulnerable Function

void vulnerable_function(char *input) {
    char local_buffer[64];  // Only 64 bytes!

    printf("Copying into 64-byte buffer...\n");
    strcpy(local_buffer, input);  // BUG: no size limit!
    printf("Copy complete\n");    // May not reach this
}

The strcpy function copies until it finds a null terminator. If input is 200 bytes, all 200 bytes get copied into a 64-byte space.

Hint 4: GDB Analysis Commands

Essential commands for your analysis:

# After loading: gdb ./corrupter core.XXXX

bt                      # See the backtrace
frame 1                 # Switch to main's frame (assuming crash in vulnerable_function)

info locals             # List local variables with values
p secret_code           # Print secret_code value
p/x secret_code         # Print as hexadecimal
p/t secret_code         # Print as binary

x/16wx &secret_code     # Examine 16 words (4-byte) around secret_code
x/64xb &buffer          # Examine buffer byte by byte
x/32wx $rbp-128         # Examine stack around frame base

# Compare before and after addresses:
p &secret_code          # Address of secret_code
p &buffer               # Address of buffer (should be lower)

Hint 5: Understanding What You See

When you examine memory, you’ll see something like:

(gdb) x/8wx &secret_code
0x7fffffffdc90:	0x41414141	0x41414141	0x41414141	0x41414141
0x7fffffffdca0:	0x41414141	0x00414141	0xffffdce0	0x55555169

Interpreting this:

0x41414141 = Four ‘A’ characters (0x41 = 65 = ‘A’ in ASCII)
This appears where secret_code (value 1337 = 0x539) should be
The pattern confirms overflow with ‘A’ characters
Address increases left-to-right, bottom-to-top
Look for where the ‘A’ pattern stops - that’s the overflow boundary

Hint 6: Full Investigation Workflow

Complete analysis session:

$ gdb ./corrupter core.1234

# 1. What happened?
(gdb) bt
#0  0x... in vulnerable_function (input=...) at corrupter.c:15
#1  0x... in main () at corrupter.c:26

# 2. What was main's state?
(gdb) frame 1
(gdb) info locals
secret_code = 1094795585
buffer = "AAAAAA..."

# 3. That number looks wrong. Check it:
(gdb) p 1094795585
$1 = 1094795585
(gdb) p/x 1094795585
$2 = 0x41414141    # AHA! That's 'AAAA'!

# 4. Examine memory around it:
(gdb) x/16xb &secret_code
0x7ffc...: 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x41 ...

# 5. Where did buffer start?
(gdb) p &buffer
$3 = 0x7ffc...   # Lower address than &secret_code

# 6. Conclusion: Overflow from vulnerable_function() overwrote main's local variable

5.8 The Interview Questions They’ll Ask

After completing this project, you’ll be ready for these questions:

“Explain how a buffer overflow works”
- Expected: Stack grows down, arrays overflow up toward higher addresses, corrupting adjacent data like saved RBP and return address
- Bonus: Mention mitigations (canaries, ASLR, NX)
“How would you debug memory corruption?”
- Expected: Generate core dump, load in GDB, backtrace to find crash location, examine memory around suspected variables, look for recognizable patterns or unexpected values
- Bonus: Mention tools like Valgrind, AddressSanitizer
“What’s the difference between print and examine in GDB?”
- Expected: print interprets data according to its declared type; examine shows raw bytes in specified format. Use examine when type information is misleading or you need to see raw memory layout.
“A program crashes in function B, but you suspect the bug is in function A. How do you investigate?”
- Expected: Use frame command to navigate to function A’s context, examine local variables, look for signs of corruption like unexpected patterns, trace data flow between functions
“What security mitigations exist against buffer overflows?”
- Expected: Stack canaries (detect overflow), ASLR (randomize addresses), NX/DEP (non-executable stack), PIE (position independent code), SafeStack, Control Flow Integrity
- Bonus: Explain why each mitigation works and what attacks it prevents
“How can you tell if a variable has been overwritten by an overflow?”
- Expected: Look for recognizable patterns (0x41414141, 0xDEADBEEF), values that don’t make sense for the variable’s purpose, addresses that point to invalid memory regions
- Bonus: Discuss stack canary trips, tool output from ASan/MSan

5.9 Books That Will Help

Topic	Book	Chapter/Section
Stack memory layout	“Dive Into Systems”	Ch. 6 (Memory Layout and Addressing)
Buffer overflow mechanics	“Hacking: The Art of Exploitation”	Ch. 2 (Programming), Ch. 3 (Exploitation)
GDB memory examination	“The Art of Debugging with GDB”	Ch. 3 (Inspecting and Changing Data)
C pointers and arrays	“Understanding and Using C Pointers”	Ch. 2, Ch. 4
x86 stack frames	“Computer Systems: A Programmer’s Perspective”	Ch. 3.7 (Procedures)
Security implications	“The Shellcoder’s Handbook”	Ch. 1-3
Safe C programming	“Secure Coding in C and C++”	Ch. 2 (Strings)

5.10 Implementation Phases

Phase 1: Basic Vulnerable Program (Day 1-2)

Goals:

Create a program that compiles and runs
Trigger a buffer overflow
Generate a core dump

Tasks:

Write corrupter.c with main() and vulnerable_function()
Implement the buffer overflow using strcpy
Compile with appropriate flags
Run and verify you get “Segmentation fault (core dumped)”
Verify core dump file exists

Checkpoint: file core.* shows “ELF 64-bit LSB core file”

Phase 2: Basic GDB Analysis (Day 3-4)

Goals:

Successfully load core dump in GDB
Get meaningful backtrace
Print variable values

Tasks:

Load core dump: gdb ./corrupter core.*
Run bt to see backtrace
Use frame to navigate
Use p to print secret_code
Verify you see unexpected value

Checkpoint: You can see that secret_code is no longer 1337

Phase 3: Memory Examination (Day 5-7)

Goals:

Use x command to examine raw memory
Understand different format specifiers
Connect memory view to variable corruption

Tasks:

Examine memory at &secret_code in hex format
Examine memory in byte, word, and giant formats
Compare addresses of variables
Map out the stack layout based on what you see
Document your findings

Checkpoint: You can explain exactly which bytes were overwritten and by what values

Phase 4: Deep Analysis (Day 8-10)

Goals:

Understand the full overflow path
Examine the vulnerable function’s stack
Calculate overflow distance

Tasks:

Examine vulnerable_function’s local_buffer
Calculate: how many bytes overflow past the buffer?
Examine the saved RBP and return address
Understand why the crash happened where it did
Experiment with different overflow sizes

Checkpoint: You can predict what happens with different overflow amounts

Phase 5: Documentation (Day 11-14)

Goals:

Create analysis script
Document findings
Prepare for variations

Tasks:

Create analyze.sh that automates common GDB commands
Write up your findings in README.md
Try different attack patterns (numbers, different characters)
Try overflowing into different variables
Summarize lessons learned

Checkpoint: You can explain the full process to someone else

5.11 Key Implementation Decisions

Decision	Options	Recommendation	Rationale
Overflow size	Small (just past buffer) vs. Large (past return addr)	Large (150-200 bytes)	More dramatic, shows full corruption potential
Attack pattern	‘A’ repeated vs. incrementing vs. mixed	‘A’ repeated	Simple, recognizable (0x41414141)
Compiler flags	With/without optimizations	No optimization (-O0)	Variables stay where declared, easier to analyze
Stack protector	Enabled vs. disabled	Disabled (-fno-stack-protector)	For learning; in production, keep it enabled
Variable types	int vs. other	int secret_code	4 bytes, easy to see full corruption
Print statements	Many vs. few	Several	Help understand execution flow

6. Testing Strategy

Test Categories

Category	Purpose	Examples
Overflow triggers crash	Verify basic functionality	Run program, get SIGSEGV
Core dump created	Verify setup	file core.* shows ELF core file
GDB loads successfully	Verify symbols	bt shows file:line info
Corruption visible	Verify overflow worked	secret_code != 1337
Pattern recognizable	Verify examination	0x41414141 visible in memory

Critical Test Cases

Minimal overflow (65 bytes):
- Just barely past buffer boundary
- May not corrupt secret_code (depends on layout)
- Use to understand exact memory layout
Medium overflow (100 bytes):
- Should corrupt some data
- May or may not corrupt return address
Large overflow (200+ bytes):
- Definitely corrupts secret_code
- May corrupt return address, causing crash on return
No overflow (baseline):
- 60-byte input (fits in 64-byte buffer)
- Program should complete normally
- secret_code should be 1337

Verification Commands

# Test 1: Core dump creation
./corrupter && echo "No crash!" || echo "Crashed (expected)"
ls -la core.* 2>/dev/null && echo "Core dump exists" || echo "No core dump"

# Test 2: GDB loads with symbols
gdb -batch -ex "bt" ./corrupter core.* 2>&1 | grep -q "corrupter.c:" && \
    echo "Symbols work" || echo "No symbols"

# Test 3: Corruption check
gdb -batch -ex "frame 1" -ex "p secret_code" ./corrupter core.* 2>&1 | \
    grep -q "1337" && echo "Not corrupted?!" || echo "Corrupted (expected)"

7. Common Pitfalls & Debugging

Frequent Mistakes

Pitfall	Symptom	Solution
Forgot `-g` flag	GDB shows `??` instead of function names	Recompile with `gcc -g`
Stack protector enabled	Crash with “stack smashing detected”	Add `-fno-stack-protector`
Core dump not created	No `core.*` file after crash	Run `ulimit -c unlimited` first
Core pattern redirects	Core file goes to systemd/apport	Check `/proc/sys/kernel/core_pattern`
Overflow too small	Variable not corrupted	Increase overflow size
Wrong frame in GDB	Can’t see expected variable	Use `frame N` to switch frames
ASLR confuses addresses	Addresses change between runs	For learning, can disable: `echo 0 > /proc/sys/kernel/randomize_va_space` (requires root)
Optimizations reorder vars	Layout doesn’t match source code order	Compile with `-O0`

Debugging Strategies

Problem: Core dump not created

# Check current limit
ulimit -c

# Check where cores go
cat /proc/sys/kernel/core_pattern

# For systemd systems, cores may go to journal:
coredumpctl list

# Simplify for learning (requires root):
sudo sysctl -w kernel.core_pattern=core.%p

Problem: GDB shows addresses but no symbol names

# Verify debug info exists
file corrupter | grep -q "not stripped" && echo "Has symbols" || echo "Stripped!"

# Verify DWARF debug info
readelf --debug-dump=info corrupter | head

# Rebuild with debug info
gcc -g -O0 -fno-stack-protector -o corrupter corrupter.c

Problem: Can’t find secret_code in GDB

# List all frames
bt

# Check which frame has secret_code
frame 0
info locals
frame 1
info locals
# Continue until you find it

# If optimization moved it:
info registers   # Value might be in register

Problem: Overflow doesn’t seem to work

// Add diagnostic output:
void vulnerable_function(char *input) {
    char local_buffer[64];
    printf("local_buffer is at: %p\n", (void*)local_buffer);
    printf("Input length: %zu\n", strlen(input));
    strcpy(local_buffer, input);
}

int main() {
    int secret_code = 1337;
    char buffer[256];
    printf("secret_code is at: %p\n", (void*)&secret_code);
    printf("buffer is at: %p\n", (void*)buffer);
    // ...
}

Quick Verification Tests

# In GDB, after loading core:

# Test 1: Can you see the backtrace?
(gdb) bt
# Should show function names and line numbers

# Test 2: Is secret_code accessible?
(gdb) frame 1
(gdb) p &secret_code
# Should show an address

# Test 3: Is it corrupted?
(gdb) p secret_code
# Should NOT be 1337

# Test 4: Can you see the pattern?
(gdb) p/x secret_code
# Should show 0x41414141 or similar

# Test 5: Can you examine memory?
(gdb) x/4wx &secret_code
# Should show hex values

8. Extensions & Challenges

Beginner Extensions

Try different patterns: Use ‘B’ (0x42), incrementing values (0x01, 0x02…), or known addresses
Print more diagnostics: Show addresses at runtime to understand layout
Variable size experimentation: Try different buffer sizes, see how much overflow is needed
Multiple variables: Add more local variables, see which ones get corrupted first

Intermediate Extensions

Corrupt the return address specifically: Calculate exact offset to return address
Add a canary manually: Implement your own stack canary check
Multi-function analysis: Chain of function calls, overflow in deepest, analyze at each level
String pattern analysis: Use patterns like “AAAABBBBCCCCDDDD” to identify exact corruption points

Advanced Extensions

Control return address: Redirect execution to a different function (in controlled environment)
Heap overflow: Similar analysis but with heap-allocated memory
Use-after-free: Create and analyze a use-after-free scenario
Integer overflow leading to buffer overflow: Trigger overflow via size calculation bug
Exploit mitigations study: Enable each mitigation, see how it changes the crash

Research Extensions

Compare to AddressSanitizer: Compile with -fsanitize=address, compare error messages
Automatic analysis script: Write Python/GDB script that automatically identifies overflow patterns
Multiple architecture comparison: Compare x86, x86-64, ARM stack layouts

9. Real-World Connections

Industry Applications

Security auditing: Finding buffer overflows in C/C++ codebases
Incident response: Analyzing crash dumps from production systems
Malware analysis: Understanding how exploits corrupt memory
Embedded systems debugging: Memory-constrained environments where corruption is common
Performance debugging: Understanding memory layout for cache optimization

Tool	Purpose	Comparison to This Project
Valgrind	Runtime memory analysis	Detects overflows as they happen, not post-mortem
AddressSanitizer	Compile-time instrumentation	More detailed errors, some runtime overhead
Electric Fence	malloc debugging	Catches heap overflows, not stack
GEF/pwndbg	GDB extensions	Enhanced GDB for exploitation work
Ghidra/IDA	Disassemblers	For when you don’t have source code

Security Relevance

Buffer overflows remain one of the top vulnerability classes:

CVE statistics show hundreds of buffer overflow CVEs annually
The MITRE Top 25 lists “Out-of-bounds Write” as #1 most dangerous software weakness
Understanding them is crucial for:
- Code review
- Secure development
- Penetration testing
- Incident response

Career Paths Using These Skills

Security Researcher: Finding and analyzing vulnerabilities
Reverse Engineer: Understanding malware and proprietary software
Kernel Developer: Debugging kernel crashes and drivers
Systems Programmer: Low-level debugging in C/C++/Rust
SRE/DevOps: Production crash analysis

10. Resources

Essential Reading

“Hacking: The Art of Exploitation” by Jon Erickson - Chapter 2-3 cover buffer overflows in depth
“Computer Systems: A Programmer’s Perspective” - Chapter 3.7 on procedures and stack frames
“Dive Into Systems” - Chapter 6 on memory layout (free online: diveintosystems.org)

Online Resources

GDB Documentation: https://sourceware.org/gdb/current/onlinedocs/gdb/
- Especially: “Examining Data” section
Smashing the Stack for Fun and Profit (Aleph One, 1996): Classic paper on buffer overflows
LiveOverflow YouTube: Excellent binary exploitation tutorials

Man Pages

man gdb          # GDB manual
man core         # Core dump format and configuration
man strcpy       # The dangerous function (note the warnings!)
man gcc          # Compiler options, including security flags

GDB Quick Reference Card

Command	Description
`bt`	Backtrace
`frame N`	Switch to frame N
`info locals`	Show local variables
`p var`	Print variable
`p/x var`	Print as hex
`p/t var`	Print as binary
`x/Nfw addr`	Examine N units of format f, size w at addr
`x/s addr`	Examine as string
`x/i addr`	Examine as instruction
`info registers`	Show register values

11. Self-Assessment Checklist

Understanding Verification

I can explain how local variables are laid out on the stack
I can explain why arrays overflow toward higher addresses on x86
I know the difference between print and examine in GDB
I understand what 0x41414141 means and why it’s useful
I can explain why strcpy is dangerous
I know what -fno-stack-protector does and why we disable it for learning
I can trace from a crash to its root cause using memory examination

Implementation Verification

My program compiles with debug symbols (no ?? in GDB backtraces)
The program crashes reproducibly
A core dump is created each time
I can load the core dump in GDB and get a backtrace
I can see secret_code has been corrupted to include my pattern
I can use x/ to examine raw memory and see the overflow bytes
I can explain the exact path from overflow to corruption

Growth Verification

I debugged at least one unexpected behavior during development
I experimented with different overflow sizes
I can navigate between stack frames and understand each context
I’m comfortable with GDB’s memory examination commands
I can explain this project’s concepts to someone else

12. Submission / Completion Criteria

Minimum Viable Completion

corrupter.c compiles and crashes as expected
Core dump is generated
Can load in GDB and get backtrace
Can demonstrate secret_code is corrupted
Can examine memory and identify overflow pattern

Full Completion

All minimum criteria met
Can examine memory at multiple locations (variables, stack pointer, frame pointer)
Can explain the exact overflow mechanism
Can calculate approximate overflow distance
Documentation of analysis process

Excellence (Going Above & Beyond)

Experiment with multiple overflow sizes and document results
Create analysis script that automates GDB commands
Study what happens when return address is corrupted
Compare behavior with/without stack protector
Implement and detect a manual canary

Summary

This project transforms you from someone who sees a crash and is stuck, to someone who sees a crash as a puzzle to solve. The skills you develop here - examining raw memory, recognizing corruption patterns, navigating stack frames - are fundamental to debugging any systems-level software.

Key Takeaways:

Memory tells the truth: Variables can lie (wrong type, corrupted value), but raw bytes don’t. The x command shows you exactly what’s there.
Patterns are your friends: Using recognizable data (0x41414141) makes corruption obvious. In real debugging, look for unexpected patterns in memory.
Location != cause: Where a program crashes is often not where the bug is. Memory examination lets you trace corruption back to its source.
Think like a detective: A core dump is a crime scene. The backtrace is the victim’s location. Memory examination reveals the weapon and how it was used.

Next Steps: After completing this project, you’re ready for:

Project 4 (Automated Crash Detective): Script these techniques for automated analysis
Project 5 (Multi-threaded Mayhem): Apply these skills to concurrent programs
Project 6 (Stripped Binary Crash): Analyze crashes when you don’t have debug symbols

This guide was expanded from LEARN_LINUX_CRASH_DUMP_ANALYSIS.md. For the complete learning path, see the README.

Project 3: The Memory Inspector

Quick Reference

1. Learning Objectives

2. Theoretical Foundation

2.1 Core Concepts

The Stack: Your Function’s Workspace

Buffer Overflow Mechanics

GDB Memory Examination Commands

2.2 Why This Matters

2.3 Historical Context

2.4 Common Misconceptions

3. Project Specification

3.1 What You Will Build

3.2 Functional Requirements

3.3 Non-Functional Requirements

3.4 Example Usage / Output

3.5 Real World Outcome

Let’s see what main’s secret_code looks like

That’s not 1337! Let’s see it in hex:

0x41 is ASCII ‘A’ - the buffer overflow pattern!

Let’s examine raw memory around secret_code:

The entire region is filled with ‘A’s (0x41)

Now let’s look at the stack to see the full picture:

We can see exactly where the overflow started and how far it went

4.4 Algorithm Overview

5. Implementation Guide

5.1 Development Environment Setup

5.2 Project Structure

5.3 The Core Question You’re Answering

5.4 Concepts You Must Understand First

5.5 Questions to Guide Your Design

5.6 Thinking Exercise

5.7 Hints in Layers

5.8 The Interview Questions They’ll Ask

5.9 Books That Will Help

5.10 Implementation Phases

Phase 1: Basic Vulnerable Program (Day 1-2)

Phase 2: Basic GDB Analysis (Day 3-4)

Phase 3: Memory Examination (Day 5-7)

Phase 4: Deep Analysis (Day 8-10)

Phase 5: Documentation (Day 11-14)

5.11 Key Implementation Decisions

6. Testing Strategy

Test Categories

Critical Test Cases

Verification Commands

7. Common Pitfalls & Debugging

Frequent Mistakes

Debugging Strategies

Quick Verification Tests

8. Extensions & Challenges

Beginner Extensions

Intermediate Extensions

Advanced Extensions

Research Extensions

9. Real-World Connections

Industry Applications

Related Tools

Security Relevance

Career Paths Using These Skills

10. Resources

Essential Reading

Online Resources

Man Pages

GDB Quick Reference Card

11. Self-Assessment Checklist

Understanding Verification

Implementation Verification

Growth Verification

12. Submission / Completion Criteria

Minimum Viable Completion

Full Completion

Excellence (Going Above & Beyond)

Summary