Project 8: Return-Oriented Programming (ROP)

Expanded deep-dive guide for Project 8 from the Binary Analysis sprint.

Quick Reference

Attribute Value
Difficulty Level 4: Expert
Time Estimate 2-3 weeks
Main Programming Language Python (pwntools)
Alternative Programming Languages Assembly understanding
Coolness Level Level 5: Pure Magic (Super Cool)
Business Potential 1. The “Resume Gold”
Knowledge Area Advanced Exploitation / Code Reuse
Software or Tool ROPgadget, ropper, pwntools
Main Book “The Shellcoder’s Handbook”

1. Learning Objectives

  1. Build a working implementation with reproducible outputs.
  2. Justify key design choices with binary-analysis principles.
  3. Produce an evidence-backed report of findings and limitations.
  4. Document hardening or next-step improvements.

2. All Theory Needed (Per-Concept Breakdown)

This project depends on concepts from the main sprint primer: loader semantics, control/data-flow recovery, runtime observation, and mitigation-aware vulnerability reasoning. Before implementation, restate the project’s core assumptions in your own words and define how you will validate them.

3. Project Specification

3.1 What You Will Build

Complex ROP chains that bypass NX protection by chaining together code snippets already in the binary.

3.2 Functional Requirements

  1. Accept the target binary/input and validate format assumptions.
  2. Produce analyzable outputs (console report and/or artifacts).
  3. Handle malformed inputs safely with explicit errors.

3.3 Non-Functional Requirements

  • Reproducibility: same input should produce equivalent findings.
  • Safety: unknown samples run only in isolated lab contexts.
  • Clarity: separate facts, hypotheses, and inferred conclusions.

3.4 Expanded Project Brief

  • File: P08-return-oriented-programming-rop.md

  • Main Programming Language: Python (pwntools)
  • Alternative Programming Languages: Assembly understanding
  • Coolness Level: Level 5: Pure Magic (Super Cool)
  • Business Potential: 1. The “Resume Gold”
  • Difficulty: Level 4: Expert
  • Knowledge Area: Advanced Exploitation / Code Reuse
  • Software or Tool: ROPgadget, ropper, pwntools
  • Main Book: “The Shellcoder’s Handbook”

What you’ll build: Complex ROP chains that bypass NX protection by chaining together code snippets already in the binary.

Why it teaches binary analysis: ROP is the foundation of modern exploitation. It demonstrates deep understanding of calling conventions and code reuse.

Core challenges you’ll face:

  • Finding gadgets → maps to instruction sequences ending in ret
  • Chaining gadgets → maps to building functionality from fragments
  • Setting up arguments → maps to calling conventions (rdi, rsi, rdx)
  • Calling system() → maps to executing /bin/sh

Resources for key challenges:

Key Concepts:

  • Gadget Types: “The Shellcoder’s Handbook” Ch. 9
  • x64 Calling Convention: System V ABI
  • Stack Pivoting: ROP Emporium tutorials

Difficulty: Expert Time estimate: 2-3 weeks Prerequisites: Project 7 (Buffer Overflow)

Real World Outcome

Deliverables:

  • Analysis output or tooling scripts
  • Report with control/data flow notes

Validation checklist:

  • Parses sample binaries correctly
  • Findings are reproducible in debugger
  • No unsafe execution outside lab ```python from pwn import *

elf = ELF(‘./target’) libc = ELF(‘./libc.so.6’) rop = ROP(elf)

Find gadgets

pop_rdi = rop.find_gadget([‘pop rdi’, ‘ret’])[0] ret = rop.find_gadget([‘ret’])[0]

Leak libc address

payload = flat( b’A’ * offset, pop_rdi, elf.got[‘puts’], # Argument: puts@GOT elf.plt[‘puts’], # Call puts to leak elf.symbols[‘main’] # Return to main for second stage )

p.sendline(payload) leaked = u64(p.recv(6).ljust(8, b’\x00’)) libc.address = leaked - libc.symbols[‘puts’]

Second stage: call system(“/bin/sh”)

bin_sh = next(libc.search(b’/bin/sh’)) system = libc.symbols[‘system’]

payload2 = flat( b’A’ * offset, ret, # Stack alignment pop_rdi, bin_sh, system )

p.sendline(payload2) p.interactive()


#### Hints in Layers
Gadget hunting:
```bash
$ ROPgadget --binary ./target | grep "pop rdi"
0x00401233 : pop rdi ; ret
$ ROPgadget --binary ./target | grep "pop rsi"
0x00401231 : pop rsi ; pop r15 ; ret

Common ROP patterns:

  1. Leak libc: Call puts(GOT_entry) to leak address
  2. Calculate libc base: leaked_addr - offset = libc_base
  3. Find /bin/sh: Search libc for “/bin/sh” string
  4. Call system: pop rdi; ret + “/bin/sh” addr + system addr

Stack alignment:

  • x64 requires 16-byte stack alignment before call
  • Add a ret gadget if system() crashes

Learning milestones:

  1. Find gadgets → Use ROPgadget or ropper
  2. Chain simple ROP → Control function arguments
  3. Leak libc → Bypass ASLR
  4. Get shell → Complete exploitation chain

The Core Question You Are Answering

How do you construct arbitrary computational logic from tiny fragments of existing code when direct code execution is impossible, and how do you chain these fragments to bypass the most sophisticated memory protection mechanisms?

This project represents the pinnacle of code-reuse attacks. You’ll learn to “program” using only code snippets (gadgets) that already exist in the binary, treating the stack as your instruction stream and gadgets as your instruction set.

Concepts You Must Understand First

1. What is a Gadget?

A gadget is a short instruction sequence ending in ret. Each gadget performs a small operation (like pop rdi; ret) and returns control, allowing you to chain gadgets together.

Gadget anatomy:
   0x401234: pop rdi          ← Useful operation
   0x401235: ret              ← Returns to next gadget

Stack layout during ROP:
   +------------------+
   | Gadget 1 addr    | ← Return here first
   | Data for gadget1 |
   | Gadget 2 addr    | ← Then return here
   | Data for gadget2 |
   | Gadget 3 addr    | ← Then return here
   | ...              |
   +------------------+

Guiding Questions:

  • Why must gadgets end in ret?
  • How does the ret instruction enable chaining?
  • What makes a gadget “useful” vs “junk”?

Book Reference: “Practical Binary Analysis” Ch. 10.2 - Code-Reuse Attacks

2. x64 Calling Convention (System V ABI)

To call functions via ROP, you must understand argument passing: RDI (1st arg), RSI (2nd), RDX (3rd), RCX (4th), R8 (5th), R9 (6th).

Guiding Questions:

  • How do you call `system(“/bin/sh”)` with ROP? (Hint: set RDI)
  • What’s the difference between x64 and x86 calling conventions?
  • Why do you need `pop rdi; ret` gadgets specifically?

Book Reference: “Low-Level Programming” Ch. 9 - Calling Conventions

3. GOT (Global Offset Table) and PLT (Procedure Linkage Table)

The GOT stores addresses of library functions (resolved at runtime). The PLT provides stubs to call them. Leaking GOT entries defeats ASLR.

Program calls printf():
   call printf@PLT  ← PLT stub

PLT stub:
   jmp [printf@GOT]  ← Jump to address in GOT

GOT entry (after first call):
   0x7ffff7a62800  ← Actual printf address in libc

Guiding Questions:

  • Why does the GOT contain real addresses but the PLT doesn’t?
  • How do you leak a GOT entry to find libc base?
  • What’s “lazy binding” and why does it matter for exploitation?

Book Reference: “Computer Systems: A Programmer’s Perspective” Ch. 7.12 - Position-Independent Code

4. Information Leaks for ASLR Bypass

Since ASLR randomizes library addresses, you must leak an address first. Common technique: call `puts(GOT_entry)` to print the address.

Guiding Questions:

  • Why leak puts or printf addresses specifically?
  • How do you calculate libc base from a leaked function address?
  • What’s a “two-stage” exploit and why is it necessary?

Book Reference: “Practical Binary Analysis” Ch. 10.3 - Randomization-Based Defenses

5. Stack Alignment Requirements

x64 requires the stack pointer (RSP) to be 16-byte aligned before executing a `call` instruction. Misalignment causes segfaults.

Guiding Questions:

  • Why does `system()` crash when called from ROP but work from ret2libc?
  • How does adding a `ret` gadget fix alignment?
  • What happens when RSP is misaligned (e.g., RSP % 16 != 0)?

Book Reference: System V ABI x86-64 specification

6. Gadget Types and Their Uses

Different gadget types serve different purposes:

  • Argument gadgets: `pop rdi; ret` (set function arguments)
  • Arithmetic gadgets: `add rax, rbx; ret` (compute values)
  • Memory gadgets: `mov [rax], rbx; ret` (write memory)
  • Control gadgets: `jmp rax` (conditional logic)

Guiding Questions:

  • Which gadget types are essential for basic exploitation?
  • How do you handle functions requiring 3+ arguments?
  • What do you do when the perfect gadget doesn’t exist?

Book Reference: “The Shellcoder’s Handbook” Ch. 9 - Return-Oriented Programming

7. Libc Database and Version Fingerprinting

Different libc versions have functions at different offsets. To find the right libc, you fingerprint it by leaking multiple addresses.

Guiding Questions:

  • Why can’t you just hardcode libc offsets?
  • How does libc-database.com help find the right version?
  • What happens if you use the wrong libc version in your exploit?

Book Reference: CTF writeups and online resources (libc.blukat.me, libc.rip)

8. Advanced ROP Techniques

Beyond basic ROP:

  • Stack pivoting: Move RSP to a controlled buffer
  • SROP (Sigreturn ROP): Use `sigreturn` to set all registers
  • JOP (Jump-Oriented Programming): Use `jmp` instead of `ret`
  • ret2csu: Use `__libc_csu_init` for arbitrary gadgets

Guiding Questions:

  • When do you need stack pivoting?
  • What makes SROP powerful (hint: it sets ALL registers)?
  • Why is `__libc_csu_init` present in every dynamically linked binary?

Book Reference: “Practical Binary Analysis” Ch. 10.2.3 - Advanced ROP

Questions to Guide Your Design

  1. How do you find gadgets when automated tools like ROPgadget fail or miss useful sequences? Consider manual searching, analyzing compiler-generated code, and understanding common instruction patterns.

  2. What’s your strategy when you need a gadget that doesn’t exist in the binary? Think about combining multiple gadgets, using library functions, or finding equivalent sequences.

  3. How would you structure a ROP chain that calls multiple functions in sequence (e.g., `mprotect()` then `shellcode()`)? Consider stack layout, argument setup, and return addresses.

  4. When you leak a libc address, how do you reliably identify which libc version is running? Think about fingerprinting multiple functions, libc databases, and offset patterns.

  5. How do you debug a ROP chain that crashes midway through execution? Consider GDB breakpoints on gadgets, stack inspection, and pwntools logging.

  6. What approach works when ASLR is enabled but you can’t find a good leak primitive? Think about partial overwrites, brute force, or other information disclosure vulnerabilities.

  7. How would you automate ROP chain generation for repeated exploitation? Consider pwntools’ ROP class, custom scripts, and chain templates.

  8. When exploiting a remote service, how do you handle the lack of direct debugging access? Think about local replication, binary analysis, and remote crash behavior.

Thinking Exercise

Before building complex ROP chains, complete these exercises:

  1. Manual Gadget Discovery: Take a simple binary and manually search for gadgets using objdump.

  2. Stack Layout Visualization: Draw the complete stack layout for a ROP chain step by step.

  3. Libc Leak Practice: Practice calculating libc base from leaked GOT entries.

  4. Building a Simple ROP Chain: Write a complete ROP chain to call write(1, buffer, 100).

The Interview Questions They’ll Ask

  1. “Explain ROP at a high level. Why is it called ‘return-oriented’?” Expected: Uses `ret` instruction to chain code snippets (gadgets). Each gadget ends with `ret`, which loads the next gadget’s address from the stack.

  2. “How do you call system(‘/bin/sh’) using ROP on x64?” Expected: Need `pop rdi; ret` to set RDI = “/bin/sh” address, then call `system@plt` or leak libc and call libc’s `system`.

  3. “What’s the difference between a gadget and regular shellcode?” Expected: Shellcode is custom assembly you inject. Gadgets are existing code fragments you reuse. ROP works when stack is non-executable (NX).

  4. “Why do you need to leak libc addresses? Can’t you just use hardcoded offsets?” Expected: ASLR randomizes libc base address on each execution. Must leak a known function’s address to calculate base.

  5. “Walk me through a two-stage ROP exploit that defeats ASLR.” Expected: Stage 1: Leak libc address (puts(GOT_entry)), return to main. Stage 2: Use leaked address to calculate libc base, call system(“/bin/sh”).

  6. “What’s stack alignment and why does system() crash in ROP but not normally?” Expected: x64 requires RSP % 16 == 0 before `call`. Normal code maintains this, but ROP might not. Fix: add `ret` gadget for alignment.

  7. “How do you find gadgets when ROPgadget doesn’t find what you need?” Expected: Manual searching with objdump, looking for unintended gadgets (instructions misaligned), using ret2csu or other universal gadgets.

  8. “Explain the GOT and PLT. How do you leak a GOT entry?” Expected: PLT stubs call functions via GOT. GOT contains actual addresses (after lazy binding). Leak: call puts(GOT_entry) to print the address.

  9. “What’s ret2csu and why is it useful?” Expected: `__libc_csu_init` function contains gadgets to control RDI, RSI, RDX. Present in all dynamically linked binaries. Provides universal gadgets.

  10. “Describe a scenario where ROP is necessary vs simpler exploitation techniques.” Expected: NX prevents shellcode execution. Stack canaries prevent simple overwrites. ASLR prevents hardcoded addresses. ROP bypasses all three.

Books That Will Help

Topic Book Chapter/Section
ROP Fundamentals “Practical Binary Analysis” Ch. 10.2 (Code-Reuse Attacks)
Advanced ROP Techniques “Practical Binary Analysis” Ch. 10.2.3 (Advanced ROP, SROP)
Calling Conventions (x64) “Low-Level Programming” Ch. 9 (Calling Conventions)
GOT/PLT Mechanism “Computer Systems: A Programmer’s Perspective” Ch. 7.12 (Position-Independent Code)
ROP Theory “The Shellcoder’s Handbook” Ch. 9 (Return-Oriented Programming)
Stack Alignment System V ABI x86-64 Specification Section 3.2.2 (The Stack Frame)
ASLR and Bypasses “Practical Binary Analysis” Ch. 10.3 (Randomization Defenses)
Dynamic Linking “Computer Systems: A Programmer’s Perspective” Ch. 7 (Linking)
Exploitation Techniques “Hacking: The Art of Exploitation” Ch. 5 (Shellcode)
Pwntools for ROP Official Pwntools Docs docs.pwntools.com/rop.html
Assembly (x64) “Low-Level Programming” Ch. 3-4 (Assembly Language)
ret2csu Technique CTF Writeups Multiple sources online
Gadget Hunting “The Shellcoder’s Handbook” Ch. 9.2 (Finding Gadgets)
Stack Pivoting “Practical Binary Analysis” Ch. 10.2.3 (Advanced Techniques)
Sigreturn-Oriented Programming Research Papers “Framing Signals—A Return to Portable Shellcode”

Common Pitfalls and Debugging

Problem 1: “Your interpretation does not match runtime behavior”

  • Why: Static analysis can hide runtime-resolved addresses, lazy binding, and input-dependent branches.
  • Fix: Reproduce the path with debugger or tracer, then compare static assumptions against live register/memory state.
  • Quick test: Run the same sample through both your static workflow and a debugger transcript, and confirm control-flow decisions align.

Problem 2: “Tool output is inconsistent across machines”

  • Why: ASLR, tool version drift, and different binary build flags (PIE, RELRO, symbols stripped) change observed addresses and metadata.
  • Fix: Pin tool versions, capture checksec/metadata, and document environment assumptions in your report.
  • Quick test: Re-run analysis in a container or VM with pinned tools and compare hashes of generated outputs.

Problem 3: “Analysis accidentally executes unsafe code”

  • Why: Dynamic workflows run binaries in host context without sufficient isolation.
  • Fix: Use disposable snapshots, no-network execution, and non-privileged users for all unknown samples.
  • Quick test: Validate isolation controls first (network disabled, snapshot active, unprivileged user), then execute sample.

Definition of Done

  • Core functionality works on reference inputs
  • Edge cases are tested and documented
  • Results are reproducible (same binary, same tools, same report output)
  • Analysis notes clearly separate observations, assumptions, and conclusions
  • Lab safety controls were applied for any dynamic execution

4. Solution Architecture

Input Artifact -> Parse/Decode -> Analysis Engine -> Validation Layer -> Report

Design each stage so intermediate artifacts are inspectable (JSON/text/notes), which makes debugging and peer review much easier.

5. Implementation Phases

Phase 1: Foundation

  • Define input assumptions and format checks.
  • Produce a minimal golden output on one known sample.

Phase 2: Core Functionality

  • Implement full analysis pass for normal cases.
  • Add validation against an external ground-truth tool.

Phase 3: Hard Cases and Reporting

  • Add malformed/edge-case handling.
  • Finalize report template and reproducibility notes.

6. Testing Strategy

  • Unit-level checks for parser/decoder helpers.
  • Integration checks against known binaries/challenges.
  • Regression tests for previously failing cases.

7. Extensions & Challenges

  • Add automation for batch analysis and comparative reports.
  • Add confidence scoring for each major finding.
  • Add export formats suitable for CI/security pipelines.

8. Production Reflection

Map your project output to a production analogue: what reliability, observability, and security controls would be required to run this continuously in an engineering organization?