Project 9: ELF/PE Loader Map Explorer

A tool that maps ELF or PE headers to a virtual memory layout diagram.

Quick Reference

Attribute	Value
Difficulty	Level 4
Time Estimate	3-4 weeks
Main Programming Language	Python or C (Alternatives: Rust, Go)
Alternative Programming Languages	Rust, Go
Coolness Level	Level 4
Business Potential	2
Prerequisites	Object Files, Linking, and Relocations, Data Representation, Memory, and Addressing
Key Topics	Object Files, Linking, and Relocations, Data Representation, Memory, and Addressing

1. Learning Objectives

By completing this project, you will:

Explain why elf/pe loader map explorer reveals key x86-64 behaviors.
Build a deterministic tool with clear, inspectable output.
Validate correctness against a golden reference output.
Connect the tool output to ABI and architecture rules.
Executables are not magic; the loader maps sections into memory with permissions and alignment.

2. All Theory Needed (Per-Concept Breakdown)

Object Files, Linking, and Relocations

Fundamentals Object files are containers for machine code, data, symbols, and relocation information. The linker merges object files into executables or shared libraries, resolving symbols and applying relocations. x86-64 systems primarily use ELF on Linux and macOS (Mach-O on macOS) and PE on Windows. The System V ABI defines the general ABI and the x86-64 psABI specifies details for ELF. Understanding sections, symbols, and relocations is required for interpreting binaries, debugging linking errors, and building tooling that inspects executables. (Sources: System V gABI, System V AMD64 ABI, Linux Foundation refspecs)

Deep Dive The object file is the bridge between assembly and execution. It holds code and data in sections, along with metadata that tells the linker how to connect references across compilation units. A symbol is a named addressable entity: a function, a global variable, or a section. Relocations are placeholders that tell the linker or loader to adjust addresses when the final layout is known. Without relocation, code would need fixed addresses and would not be portable or shareable.

In ELF, sections such as .text, .data, and .bss hold code and data. The section headers describe offsets, sizes, and flags. The symbol table maps symbol names to section offsets and attributes. Relocation entries refer to symbols and specify how to patch the code or data. The relocation types indicate whether a relocation is absolute, PC-relative, or uses a GOT/PLT indirection. The x86-64 psABI defines these relocation types and their semantics, which is essential for interpreting dynamic linking and position-independent code.

The linker performs symbol resolution: it decides which definition of a symbol to use and patches references accordingly. Static linking resolves all symbols at link time. Dynamic linking defers some resolution to runtime, using the dynamic loader and data structures such as the Global Offset Table (GOT) and Procedure Linkage Table (PLT). The GOT holds addresses of global symbols and is updated by the loader. The PLT provides a stub that jumps through the GOT, enabling lazy binding. This mechanism is central to shared libraries and is a common target for debugging and security analysis.

PE on Windows uses a different layout but similar ideas: sections, import tables, and relocation entries. The import table lists external functions that must be resolved at load time. Base relocations allow the loader to rebase the executable if it cannot be loaded at the preferred address. While the details differ, the mental model is the same: object files are templates, and the loader fills in the addresses.

Relocations also matter for reverse engineering. If you see a relocation against a symbol, you know that the code depends on that symbol even if the address is not fixed. This is how tools recover call graphs and identify external dependencies. It is also how you can locate jump tables, vtables, and other data-driven control flow constructs.

Finally, debugging symbols and unwind info live alongside code in the object file. These sections are optional but invaluable for debugging and profiling. Understanding them helps you build tools that can show file/line information, variable locations, and call stacks, even in optimized binaries.

How this fits on projects

Projects 9 and 10 are focused on ELF/PE parsing and relocation resolution.
Project 6 uses unwind metadata to visualize stack frames.

Definitions & key terms

Object file: Compiled code and metadata before linking.
Section: A region of an object file with a specific purpose.
Symbol: Named reference to code or data.
Relocation: A patch applied by the linker or loader.
GOT/PLT: Indirection tables for dynamic linking.

Mental model diagram

SOURCE -> OBJECT (.o) -> LINKER -> EXECUTABLE -> LOADER -> RUNNING

OBJECT:
  [ .text ] [ .data ] [ .bss ] [ .symtab ] [ .rel.* ]

LINKER:
  resolve symbols + apply relocations

LOADER:
  map segments + resolve dynamic relocations

How it works

Compiler/assembler emits object file with symbols and relocations.
Linker merges sections and resolves symbols.
Linker applies relocations or marks them for runtime.
Loader maps segments and resolves dynamic relocations.

Invariants and failure modes:

Invariant: Relocations reference valid symbols.
Failure: Missing symbols cause link errors or runtime crashes.
Invariant: Section permissions match content (code vs data).
Failure: Incorrect permissions can cause execution faults.

Minimal concrete example (pseudo-structure)

# PSEUDOCODE ONLY
SECTION .text:
  CALL SYMBOL_F
RELOCATION:
  at .text+0x10 -> SYMBOL_F (PC_REL)

Common misconceptions

“Linking is just concatenation.” It includes symbol resolution and relocation.
“GOT/PLT is only for performance.” It is required for dynamic linking.
“Object files are the final executable.” They are templates.

Check-your-understanding questions

Why do relocation entries exist at all?
What problem does the PLT solve?
How does a loader differ from a linker?

Check-your-understanding answers

Addresses are not final until link or load time.
It enables lazy binding of external functions.
The linker combines objects; the loader maps and relocates at runtime.

Real-world applications

Diagnosing link errors and symbol conflicts
Reverse engineering binary dependencies
Building binary inspection tools

Where you will apply it Projects 6, 9, 10

References

System V ABI / gABI (Linux Foundation refspecs)
System V AMD64 ABI Draft 0.99.7
“Computer Systems: A Programmer’s Perspective” by Bryant and O’Hallaron - Ch. 7

Key insights Relocations and symbols are the invisible glue that makes binaries runnable.

Summary Understanding object files and linking turns binaries from opaque blobs into structured systems.

Homework/Exercises to practice the concept

Sketch the sections of a minimal object file and label their purpose.
Explain how a call to an external function is resolved.

Solutions to the homework/exercises

Include .text, .data, .bss, .symtab, and relocation sections.
Linker or loader patches a placeholder using relocation info.
Data Representation, Memory, and Addressing

Fundamentals x86-64 is a byte-addressed, little-endian architecture. Data representation determines how values appear in memory, how loads and stores reconstruct those values, and how alignment affects performance. Memory addressing is not just “base + offset”; it is a rich set of forms including base, index, scale, and displacement, plus RIP-relative addressing in 64-bit mode. These addressing forms are part of the ISA and are a primary tool for compilers. Virtual memory adds another layer: the addresses you see in registers are virtual, translated by page tables configured by the OS. When you write or analyze assembly, you are always navigating both representation and translation. Official architecture references and ABI specifications describe these addressing forms and constraints. (Sources: Intel SDM, Microsoft x64 architecture docs)

Deep Dive Data representation is the mapping between abstract values and physical bytes. On x86-64, integers are typically two’s complement and stored in little-endian order. That means the least significant byte sits at the lowest memory address. When you inspect memory dumps, the order will appear reversed relative to the human-readable hex. This matters for debugging and binary analysis; it also matters for writing correct parsing and serialization logic.

Memory addressing is a key differentiator between x86-64 and many simpler ISAs. The architecture supports effective addresses of the form base + index * scale + displacement, where scale can be 1, 2, 4, or 8. This lets the CPU calculate addresses for arrays and structures in a single instruction, which is why compiler output often uses complex addressing instead of explicit multiply or add instructions. In long mode, RIP-relative addressing is widely used for position-independent code; it allows the instruction stream to refer to nearby constants and jump tables without absolute addresses. That is why you will see references relative to RIP rather than absolute pointers in modern binaries.

Virtual memory is the next layer of meaning. The addresses in registers are virtual; they are translated to physical addresses using a page table hierarchy. As a result, two different processes can have the same virtual address mapping to different physical memory. The OS enforces protection and isolation through page permissions. When you read assembly, you see the virtual addresses. The mapping is invisible unless you consult page tables or OS introspection tools, which is why memory corruption bugs can appear non-deterministic; they might read valid memory but the wrong mapping.

Alignment is another subtlety. Many instructions perform better when data is aligned to its natural width (for example, 8-byte aligned for 64-bit values). Misaligned loads are supported in x86-64 but can be slower or cause extra microarchitectural work. ABI conventions often require stack alignment to 16 bytes at call boundaries, which ensures that SIMD operations and stack-based data are aligned. This alignment rule is part of the ABI, not just a performance hint.

Addressing modes also influence instruction encoding. The ModR/M and SIB bytes encode the base, index, scale, and displacement. Some combinations are invalid or have special meaning (for example, certain base/index fields imply RIP-relative addressing or a displacement-only form). Understanding this encoding is critical for building decoders and for interpreting bytes in memory. It is also how you can verify that a disassembler is correct: the addressing mode can be inferred from the encoding and compared to the textual rendering.

Finally, consider how data representation affects control flow and calling conventions. Arguments passed by reference are simply addresses; the ABI does not enforce type. That means assembly must interpret the bytes correctly, or the program will behave incorrectly even if the instruction sequence is “valid.” This is where assembly becomes a discipline: you must know what the bytes mean, and that meaning is not written anywhere except in the ABI and the program’s logic.

How this fits on projects

Projects 2-4 are explicitly about effective address calculation and RIP-relative forms.
Projects 9-10 require precise understanding of data layout and alignment inside ELF/PE sections.

Definitions & key terms

Little-endian: Least significant byte at lowest address.
Effective address: The computed address used by a memory instruction.
RIP-relative: Addressing relative to the instruction pointer.
Virtual memory: The address space seen by a process, mapped to physical memory.
Alignment: Address boundary that improves correctness or performance.

Mental model diagram

VALUE -> BYTES -> VIRTUAL ADDRESS -> PAGE TABLE -> PHYSICAL ADDRESS

      +---------+          +------------------+
      |  Value  |  encode  |  Byte Sequence   |
      +---------+          +---------+--------+
                                    |
                                    v
                         +----------------------+
                         |  Effective Address   |
                         | base + index*scale + |
                         |      displacement    |
                         +----------+-----------+
                                    |
                                    v
                         +----------------------+
                         |   Virtual Address    |
                         +----------+-----------+
                                    |
                                    v
                         +----------------------+
                         |   Page Translation   |
                         +----------+-----------+
                                    |
                                    v
                         +----------------------+
                         |   Physical Address   |
                         +----------------------+

How it works

Program computes effective address from base/index/scale/disp.
CPU uses that effective address as a virtual address.
MMU translates virtual to physical using page tables.
Data is loaded or stored in little-endian byte order.

Invariants and failure modes:

Invariant: Effective address is computed before translation.
Failure: Misinterpreting endianness yields wrong values.
Invariant: ABI defines alignment at call boundaries.
Failure: Misalignment can break SIMD assumptions or slow down code.

Minimal concrete example (pseudo-assembly, not real code)

# PSEUDOCODE ONLY
# Compute address of element i in an array of 8-byte elements
EFFECTIVE_ADDRESS = BASE_PTR + INDEX * 8 + OFFSET
LOAD64 REG_X, [EFFECTIVE_ADDRESS]

Common misconceptions

“x86-64 is big-endian.” It is little-endian by default.
“All addresses are physical.” User code uses virtual addresses.
“Alignment is optional.” It is required by ABI for some operations.

Check-your-understanding questions

Why does little-endian matter when reading a hexdump?
What is the difference between effective and virtual address?
Why do compilers use base+index*scale addressing?

Check-your-understanding answers

The byte order is reversed relative to human-readable hex.
Effective is computed by the instruction; virtual is then translated.
It encodes array indexing in a single instruction.

Real-world applications

Debugging pointer arithmetic errors
Building instruction decoders and disassemblers
Understanding how compilers lay out data

Where you will apply it Projects 2, 3, 4, 9, 10

References

Intel 64 and IA-32 Architectures Software Developer’s Manual (Intel)
Microsoft x64 architecture documentation
“Computer Systems: A Programmer’s Perspective” by Bryant and O’Hallaron - Ch. 3

Key insights Memory is not just bytes; it is a layered mapping between representation and address translation.

Summary Effective addressing and data layout are the glue between values in your head and bytes in memory.

Homework/Exercises to practice the concept

Convert a 64-bit integer into its little-endian byte sequence.
Compute effective addresses for an array with different indices.

Solutions to the homework/exercises

List the bytes from least significant to most significant.
Use base + index * element_size + offset.

3. Project Specification

3.1 What You Will Build

A tool that maps ELF or PE headers to a virtual memory layout diagram.

Why this teaches x86-64: Executables are not magic; the loader maps sections into memory with permissions and alignment.

Included:

Deterministic CLI output for a fixed input
Clear mapping between inputs and architectural meaning
A small test suite with edge cases

Excluded:

Full compiler or full disassembler coverage
Production-grade UI or packaging

3.2 Functional Requirements

Deterministic Output: Same input yields identical output.
Architecture-Aware: Output references ABI/ISA rules where relevant.
Validation Mode: Provide a compare mode against a golden output.

3.3 Non-Functional Requirements

Performance: Fast enough for small inputs and interactive use.
Reliability: Handles malformed inputs with clear errors.
Usability: Outputs are readable and documented.

3.4 Example Usage / Output

$ x64map demo.elf

SEGMENT MAP
.text  VA=0x400000  SIZE=0x1000  PERM=R-X
.rodata VA=0x401000 SIZE=0x0800  PERM=R--
.data  VA=0x402000  SIZE=0x0400  PERM=RW-
.bss   VA=0x403000  SIZE=0x0200  PERM=RW-

3.5 Data Formats / Schemas / Protocols

Input format: line-oriented text or hex bytes (documented in README)
Output format: stable, human-readable report with labeled fields

3.6 Edge Cases

Empty input or missing fields
Invalid numeric values or malformed hex
Inputs that exercise maximum/minimum bounds

3.7 Real World Outcome

This section is your golden reference. Match it exactly.

3.7.1 How to Run (Copy/Paste)

Build: (if needed) make or equivalent
Run: P09-elf-pe-loader-map-explorer with sample input
Working directory: project root

3.7.2 Golden Path Demo (Deterministic)

Run with the provided demo input and confirm output matches the transcript.

3.7.3 If CLI: exact terminal transcript

$ x64map demo.elf

SEGMENT MAP
.text  VA=0x400000  SIZE=0x1000  PERM=R-X
.rodata VA=0x401000 SIZE=0x0800  PERM=R--
.data  VA=0x402000  SIZE=0x0400  PERM=RW-
.bss   VA=0x403000  SIZE=0x0200  PERM=RW-

4. Solution Architecture

4.1 High-Level Design

INPUT -> PARSER -> MODEL -> RENDERER -> REPORT

4.2 Key Components

Component	Responsibility	Key Decisions
Parser	Turn input into structured records	Strict vs permissive parsing
Model	Apply ISA/ABI rules	Deterministic state transitions
Renderer	Produce readable output	Stable formatting

4.4 Data Structures (No Full Code)

Record: holds one instruction/event with decoded fields
State: represents register/flag or address state
Report: list of formatted output lines

4.4 Algorithm Overview

Key Algorithm: Parse and Evaluate

Parse input into records.
Apply rules to update state.
Render the state and summary output.

Complexity Analysis:

Time: O(n) over input records
Space: O(n) for report output

5. Implementation Guide

5.1 Development Environment Setup

# Ensure basic tools are installed
# build-essential or clang, plus objdump/readelf if needed

5.2 Project Structure

project-root/
├── src/
│   ├── main.*
│   ├── parser.*
│   └── model.*
├── tests/
│   └── test_cases.*
└── README.md

5.3 The Core Question You’re Answering

How does the loader transform an on-disk file into a running memory image?

5.4 Concepts You Must Understand First

ELF/PE headers
- What do program headers and section headers represent?
- Book Reference: “Computer Systems: A Programmer’s Perspective” - Ch. 7
Virtual memory
- How are segments mapped into memory?
- Book Reference: “Operating Systems: Three Easy Pieces” - Ch. 13

5.5 Questions to Guide Your Design

Format choice
- Will you support ELF only or both ELF and PE?
- How will you detect file type?
Visualization
- How will you render the memory map?
- How will you show permissions and alignment gaps?

5.6 Thinking Exercise

Segment Layout

Draw a memory map with .text, .rodata, .data, and .bss. Show which are executable and which are writable.

Questions to answer:

Why should code be non-writable?
Why is .bss not stored on disk?

5.7 The Interview Questions They’ll Ask

“What is the difference between ELF sections and segments?”
“Why does the loader care about program headers?”
“What is the purpose of .bss?”
“How do permissions affect security?”
“Why can executables be relocated in memory?”

5.8 Hints in Layers

Hint 1: Starting Point Parse only the minimal header fields you need for mapping.

Hint 2: Next Level Build a table of segments with virtual address, size, and permissions.

Hint 3: Technical Details Align segments to page boundaries in your visualization.

Hint 4: Tools/Debugging Compare with readelf -l output for a known binary.

5.9 Books That Will Help

Topic	Book	Chapter
Linking and loading	“Computer Systems: A Programmer’s Perspective”	Ch. 7
Virtual memory	“Operating Systems: Three Easy Pieces”	Ch. 13

5.10 Implementation Phases

Phase 1: Foundation (2-3 days)

Goals:

Parse input format
Produce a minimal output Tasks:
1. Define input grammar and example files.
2. Implement a minimal parser and renderer. Checkpoint: Golden output matches a small input.

Phase 2: Core Functionality (1 week)

Goals:

Implement full rule set
Add validation and errors Tasks:
1. Implement rule engine for core cases.
2. Add error handling for invalid inputs. Checkpoint: All core tests pass.

Phase 3: Polish & Edge Cases (2-3 days)

Goals:

Add edge-case coverage
Improve output readability Tasks:
1. Add edge-case tests.
2. Refine output formatting and summary. Checkpoint: Output matches golden transcript for all cases.

5.11 Key Implementation Decisions

Decision	Options	Recommendation	Rationale
Input format	Text, JSON	Text	Easiest to audit and diff
Output format	Plain text, JSON	Plain text	Matches CLI tooling

6. Testing Strategy

6.1 Test Categories

Category	Purpose	Examples
Unit Tests	Validate parsing and rule application	Valid/invalid inputs
Integration Tests	End-to-end output comparison	Golden transcripts
Edge Case Tests	Stress unusual inputs	Empty input, max values

6.2 Critical Test Cases

Minimal Input: One record, verify output.
Boundary Values: Largest/smallest values.
Malformed Input: Ensure clean error messages.

6.3 Test Data

INPUT: sample_min.txt
EXPECTED: matches golden transcript

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

Pitfall	Symptom	Solution
Wrong assumptions	Output mismatches	Re-read ABI/ISA rules
Off-by-one parsing	Missing fields	Add explicit length checks
Ambiguous output	Hard to verify	Add labels and separators

Project-specific pitfalls

Problem 1: “Segments overlap or appear out of order”

Why: Misinterpreting file offsets vs virtual addresses.
Fix: Use program headers for mapping, not section headers.
Quick test: Compare with loader output for a simple binary.

7.2 Debugging Strategies

Golden diffing: Use diff to compare outputs line by line.
State logging: Print intermediate state after each step.

7.3 Performance Traps

Avoid over-optimizing; correctness and determinism matter most.

8. Extensions & Challenges

8.1 Beginner Extensions

Add a new input case and golden output
Add a summary line with counts

8.2 Intermediate Extensions

Add JSON output mode
Add validation warnings for suspicious inputs

8.3 Advanced Extensions

Support additional ABI or instruction variants
Integrate with a real binary to collect inputs

9. Real-World Connections

9.1 Industry Applications

Profilers and tracers: Use similar decoding and state models.
Security analysis: Use precise ABI knowledge to interpret crashes.

objdump: reference tool for binary inspection.
llvm-objdump: LLVM-based disassembly and inspection.

9.3 Interview Relevance

ABI and calling conventions are common systems interview topics.
Explaining decoding and linking demonstrates low-level fluency.

10. Resources

10.1 Essential Reading

Intel 64 and IA-32 Architectures Software Developer’s Manual - ISA reference
System V AMD64 ABI Draft 0.99.7 - calling convention rules

10.2 Video Resources

Vendor and university lectures on x86-64 and ABIs (search official channels)

Project 9: ELF/PE Loader Map Explorer

Quick Reference

1. Learning Objectives

2. All Theory Needed (Per-Concept Breakdown)

Object Files, Linking, and Relocations

Data Representation, Memory, and Addressing

3. Project Specification

3.1 What You Will Build

3.2 Functional Requirements

3.3 Non-Functional Requirements

3.4 Example Usage / Output

3.5 Data Formats / Schemas / Protocols

3.6 Edge Cases

3.7 Real World Outcome

3.7.1 How to Run (Copy/Paste)

3.7.2 Golden Path Demo (Deterministic)

3.7.3 If CLI: exact terminal transcript

4. Solution Architecture

4.1 High-Level Design

4.2 Key Components

4.4 Data Structures (No Full Code)

4.4 Algorithm Overview

5. Implementation Guide

5.1 Development Environment Setup

5.2 Project Structure

5.3 The Core Question You’re Answering

5.4 Concepts You Must Understand First

5.5 Questions to Guide Your Design

5.6 Thinking Exercise

5.7 The Interview Questions They’ll Ask

5.8 Hints in Layers

5.9 Books That Will Help

5.10 Implementation Phases

Phase 1: Foundation (2-3 days)

Phase 2: Core Functionality (1 week)

Phase 3: Polish & Edge Cases (2-3 days)

5.11 Key Implementation Decisions

6. Testing Strategy

6.1 Test Categories

6.2 Critical Test Cases

6.3 Test Data

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

7.2 Debugging Strategies

7.3 Performance Traps

8. Extensions & Challenges

8.1 Beginner Extensions

8.2 Intermediate Extensions

8.3 Advanced Extensions

9. Real-World Connections

9.1 Industry Applications

9.2 Related Open Source Projects

9.3 Interview Relevance

10. Resources

10.1 Essential Reading

10.2 Video Resources