← Back to all projects

HYPERVISOR VIRTUALIZATION DEEP DIVE PROJECTS

Hypervisor & Virtualization Deep Dive Projects

Goal: Deeply understand how hypervisors work, from basic CPU emulation to hardware-assisted virtualization, culminating in building your own VM system like QEMU.

---

Core Concepts You’ll Master

Before diving into projects, here’s what you need to understand about the virtualization landscape:

The Virtualization Stack

┌─────────────────────────────────────────────────────────────┐
│                    Guest Applications                        │
├─────────────────────────────────────────────────────────────┤
│                    Guest Operating System                    │
├─────────────────────────────────────────────────────────────┤
│                Virtual Hardware (Emulated)                   │
│   ┌─────────┬─────────┬─────────┬─────────┬─────────┐       │
│   │ vCPU    │ vMemory │ vDisk   │ vNIC    │ vUSB    │       │
├───┴─────────┴─────────┴─────────┴─────────┴─────────┴───────┤
│                  Hypervisor / VMM                            │
│   ┌──────────────────┬──────────────────────────────┐       │
│   │ CPU Virtualization│ Device Emulation            │       │
│   │ (VT-x/Software)  │ (QEMU-style)                │       │
│   └──────────────────┴──────────────────────────────┘       │
├─────────────────────────────────────────────────────────────┤
│                    Host OS (optional)                        │
├─────────────────────────────────────────────────────────────┤
│                    Physical Hardware                         │
│   ┌─────────┬─────────┬─────────┬─────────┬─────────┐       │
│   │ CPU     │ RAM     │ Storage │ NIC     │ USB     │       │
│   └─────────┴─────────┴─────────┴─────────┴─────────┘       │
└─────────────────────────────────────────────────────────────┘

QEMU Clarified

QEMU is NOT a hypervisor by itself — it’s an emulator:

Mode	What It Does	Speed	Use Case
QEMU alone	Pure software emulation via TCG (Tiny Code Generator)	Slow (~10-100x slower)	Cross-architecture (ARM on x86)
QEMU + KVM	QEMU handles devices, KVM handles CPU/memory via VT-x	Near-native	Same-architecture virtualization
QEMU + Xen	QEMU provides device models for Xen guests	Near-native	Enterprise virtualization

Hypervisor Types

Type	Description	Examples
Type 1 (Bare-metal)	Runs directly on hardware, no host OS	VMware ESXi, Xen, Hyper-V, KVM*
Type 2 (Hosted)	Runs on top of a host OS	VirtualBox, VMware Workstation, QEMU

*KVM is technically a kernel module, making Linux itself the hypervisor.

CPU Virtualization Techniques

Technique	How It Works	Performance	Complexity
Interpretation	Fetch-decode-execute each instruction in software	Very slow	Simple
Binary Translation	Translate blocks of guest code to host code (JIT)	Moderate	Complex
Trap-and-Emulate	Run guest directly, trap on privileged instructions	Fast (if possible)	Moderate
Hardware-Assisted (VT-x/AMD-V)	CPU has special VMX mode for guests	Near-native	Complex setup, simple execution

---

Project Progression Path

Level 1: Interpreter Emulators (Understand the basics)
    │
    ├── Project 1: CHIP-8 Interpreter
    ├── Project 2: Simple RISC CPU Emulator
    │
Level 2: Binary Translation & JIT (How QEMU's TCG works)
    │
    ├── Project 3: Basic Block Translator
    ├── Project 4: Simple JIT Compiler
    │
Level 3: Memory Virtualization (The address space illusion)
    │
    ├── Project 5: Shadow Page Table Simulator
    ├── Project 6: Memory Mapper with Protection
    │
Level 4: Device Emulation (Virtual hardware)
    │
    ├── Project 7: Virtual Serial Port (UART)
    ├── Project 8: Virtual Block Device
    ├── Project 9: Virtual Network Interface
    │
Level 5: Hardware-Assisted Virtualization (VT-x/AMD-V)
    │
    ├── Project 10: VMX Capability Explorer
    ├── Project 11: Minimal VT-x Hypervisor
    ├── Project 12: EPT Memory Virtualization
    │
Level 6: Integration (Full VM System)
    │
    ├── Project 13: Mini-QEMU Clone
    ├── Project 14: KVM Userspace Client
    │
Level 7: Capstone
    │
    └── Project 15: Complete Type-2 Hypervisor

---

Project 1: CHIP-8 Interpreter Emulator

• File: HYPERVISOR_VIRTUALIZATION_DEEP_DIVE_PROJECTS.md
• Main Programming Language: C
• Alternative Programming Languages: Rust, Go, Zig
• Coolness Level: Level 3: Genuinely Clever
• Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
• Difficulty: Level 1: Beginner (The Tinkerer)
• Knowledge Area: CPU Emulation / Instruction Decoding
• Software or Tool: CHIP-8 Emulator
• Main Book: “Computer Systems: A Programmer’s Perspective” by Bryant & O’Hallaron

What you’ll build: A complete emulator for the CHIP-8 virtual machine that runs classic games like Pong, Tetris, and Space Invaders.

Why it teaches virtualization: CHIP-8 is the “Hello World” of emulation. It has only 35 instructions, 16 registers, and 4KB of memory. This forces you to understand the fundamental emulation loop: fetch → decode → execute. Every hypervisor, from QEMU to VMware, uses this same conceptual model for software emulation.

Core challenges you’ll face:

• Instruction fetch cycle → Understanding the program counter and memory layout
• Opcode decoding → Parsing binary instruction formats (foundational for any CPU emulation)
• Register state management → Maintaining CPU state between instructions
• Timer synchronization → Matching emulated time to real time (critical for any VM)
• Input/output emulation → Mapping virtual keyboard to host input

Key Concepts:

• Fetch-Decode-Execute Cycle: “Computer Systems: A Programmer’s Perspective” Chapter 4 - Bryant & O’Hallaron
• Instruction Encoding: “Write Great Code, Volume 2” Chapter 3 - Randall Hyde
• State Machine Design: “Code: The Hidden Language” Chapter 17 - Charles Petzold

Difficulty: Beginner Time estimate: 1 week Prerequisites: Basic C, understanding of binary/hex

Real world outcome:

$ ./chip8 roms/PONG.ch8
┌────────────────────────────────────┐
│                                    │
│  ██                          ██    │
│  ██            ●             ██    │
│  ██                          ██    │
│                                    │
└────────────────────────────────────┘
Score: Player 1: 3  Player 2: 2

You’ll have a working game emulator. Press keys, see the paddle move, watch the ball bounce. This is virtualization in its simplest form.

Implementation Hints: The CHIP-8 has a simple memory map: 0x000-0x1FF is reserved (originally for the interpreter), 0x200-0xFFF is program space. Your main loop should:

1. Read 2 bytes from memory at PC (program counter)
2. Decode the 16-bit opcode (first nibble determines instruction type)
3. Execute the instruction (modify registers, memory, or PC)
4. Decrement timers at 60Hz
5. Render display if draw flag is set

The key insight: you’re simulating a CPU by maintaining its state (registers, PC, stack) and interpreting what each instruction should do.

Learning milestones:

1. Opcodes 0x00E0 and 0x1NNN work → You understand fetch-decode-execute
2. All 35 opcodes implemented → You’ve internalized instruction set architecture
3. Games run at correct speed → You understand timing and synchronization
4. Multiple ROMs work correctly → Your emulator is robust and complete

---

Project 2: Simple RISC CPU Emulator

• File: HYPERVISOR_VIRTUALIZATION_DEEP_DIVE_PROJECTS.md
• Main Programming Language: C
• Alternative Programming Languages: Rust, C++, Zig
• Coolness Level: Level 4: Hardcore Tech Flex
• Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
• Difficulty: Level 2: Intermediate (The Developer)
• Knowledge Area: CPU Architecture / ISA Design
• Software or Tool: CPU Emulator
• Main Book: “Computer Organization and Design RISC-V Edition” by Patterson & Hennessy

What you’ll build: An emulator for a subset of RISC-V (RV32I) that can run simple compiled programs, with a debugger interface showing register state and memory.

Why it teaches virtualization: RISC-V is the real deal—a modern instruction set used in production. Emulating it teaches you how real CPUs work: the register file, ALU operations, memory addressing modes, and control flow. This is exactly what QEMU does with its TCG when emulating a target architecture.

Core challenges you’ll face:

• Instruction decoding with multiple formats → R-type, I-type, S-type, B-type, U-type, J-type
• Memory-mapped I/O → How devices communicate with CPU
• Exception handling → What happens on invalid instructions or memory access
• ABI compliance → Following calling conventions so compiled code works
• ELF loading → Parsing executable format to load programs

Key Concepts:

• RISC-V ISA: “Computer Organization and Design RISC-V Edition” Chapters 2-4 - Patterson & Hennessy
• Instruction Formats: RISC-V Specification - RISC-V Foundation
• ELF Format: “Practical Binary Analysis” Chapter 2 - Dennis Andriesse
• Memory-Mapped I/O: “Computer Systems: A Programmer’s Perspective” Chapter 6 - Bryant & O’Hallaron

Difficulty: Intermediate Time estimate: 2-3 weeks Prerequisites: Project 1, understanding of assembly basics

Real world outcome:

$ riscv64-unknown-elf-gcc -march=rv32i -mabi=ilp32 -o hello hello.c
$ ./rv32emu hello
Hello from RISC-V!

$ ./rv32emu --debug hello
PC: 0x00010074  Instruction: 0x00a00513  ADDI x10, x0, 10
Registers:
  x0=0x00000000  x1=0x000100a4  x2=0x7ffffff0  x3=0x00000000
  x10=0x0000000a x11=0x00000000 x12=0x00000000 ...
> step
PC: 0x00010078  Instruction: 0x00000073  ECALL

You can compile C programs and run them on YOUR emulator. You can step through instructions and watch registers change.

Implementation Hints: Start with just the base integer instructions (RV32I - about 40 instructions). The instruction encoding is elegantly regular—once you decode the opcode (bits 0-6), funct3 (bits 12-14), and funct7 (bits 25-31), you can determine the exact instruction.

Key insight: The CPU is just a state machine. Each cycle you:

1. Fetch instruction from memory[PC]
2. Decode format and extract operands
3. Execute ALU operation
4. Write back to register file
5. Update PC (normally PC+4, or branch target)

For I/O, memory-map a UART at 0x10000000. When guest writes to that address, print the character to host stdout.

Learning milestones:

1. ADD, SUB, AND, OR work → You understand ALU operations
2. Branches work → You understand control flow and PC manipulation
3. Memory load/store works → You understand address translation concepts
4. “Hello World” runs → Your emulator is functional
5. GCC-compiled programs run → You’re ABI compliant

---

Project 3: Basic Block Binary Translator

• File: HYPERVISOR_VIRTUALIZATION_DEEP_DIVE_PROJECTS.md
• Main Programming Language: C
• Alternative Programming Languages: C++, Rust
• Coolness Level: Level 5: Pure Magic (Super Cool)
• Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
• Difficulty: Level 3: Advanced (The Engineer)
• Knowledge Area: Binary Translation / JIT Compilation
• Software or Tool: Binary Translator
• Main Book: “Engineering a Compiler” by Cooper & Torczon

What you’ll build: A static binary translator that converts RISC-V basic blocks to x86-64 machine code, creating an executable that runs natively.

Why it teaches virtualization: This is how QEMU’s TCG (Tiny Code Generator) works. Instead of interpreting each instruction (slow), you translate blocks of guest code to host code once, then execute natively. Understanding this is key to understanding why QEMU can emulate different architectures with reasonable performance.

Core challenges you’ll face:

• Basic block identification → Finding boundaries where branches occur
• Register mapping → Guest registers to host registers
• Instruction selection → Choosing optimal host instructions for guest operations
• Code emission → Generating valid x86-64 machine code bytes
• Calling convention bridging → When guest calls differ from host ABI

Key Concepts:

• Basic Blocks and CFG: “Engineering a Compiler” Chapter 8 - Cooper & Torczon
• Instruction Selection: “Engineering a Compiler” Chapter 11 - Cooper & Torczon
• x86-64 Encoding: “Modern X86 Assembly Language Programming” - Daniel Kusswurm
• Dynamic Translation: Introduction to Dynamic Recompilation - Zenogais

Difficulty: Advanced Time estimate: 3-4 weeks Prerequisites: Projects 1-2, x86-64 assembly knowledge

Real world outcome:

$ ./translator fibonacci.rv32 -o fibonacci.native
Translating 47 basic blocks...
Block 0x1000: 12 instructions → 8 x86 instructions
Block 0x1024: 5 instructions → 4 x86 instructions
...
Generated 1,247 bytes of x86-64 code

$ ./fibonacci.native
Fibonacci(30) = 832040
Time: 0.003s

$ ./interpreter fibonacci.rv32
Fibonacci(30) = 832040
Time: 0.892s

Your translator produces code that runs 100-300x faster than interpretation!

Implementation Hints: Start simple: translate to a C file first, then compile. This “transpiler” approach lets you debug the translation logic without worrying about machine code encoding.

Once that works, emit actual x86-64 bytes:

• mov rax, imm64 = 48 B8 + 8 bytes
• add rax, rbx = 48 01 D8
• ret = C3

Map RISC-V registers to x86 registers: x10-x17 (arguments) → rdi, rsi, rdx, rcx, r8, r9. Spill the rest to a memory array.

Key insight: A basic block has one entry point and one exit point. Translate the whole block, then patch up the exit to jump to the next translated block (or back to the translator for new blocks).

Learning milestones:

1. Single block translates correctly → You understand instruction mapping
2. Multiple blocks chain together → You understand control flow translation
3. Loops work → You’ve handled backward branches
4. 10x speedup achieved → Your translation is effective
5. 100x speedup achieved → You’ve optimized register allocation

---

Project 4: Simple JIT Compiler for a Bytecode VM

• File: HYPERVISOR_VIRTUALIZATION_DEEP_DIVE_PROJECTS.md
• Main Programming Language: C
• Alternative Programming Languages: C++, Rust
• Coolness Level: Level 5: Pure Magic (Super Cool)
• Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
• Difficulty: Level 3: Advanced (The Engineer)
• Knowledge Area: JIT Compilation / Dynamic Code Generation
• Software or Tool: JIT Compiler
• Main Book: “Language Implementation Patterns” by Terence Parr

What you’ll build: A JIT compiler for a simple stack-based bytecode (like a tiny Java VM) that compiles hot paths to native x86-64 code at runtime.

Why it teaches virtualization: This is dynamic binary translation—the heart of QEMU’s TCG. You’ll understand how to generate machine code at runtime, manage executable memory, and switch between interpreted and compiled execution.

Core challenges you’ll face:

• Hot path detection → Identifying frequently-executed code worth compiling
• Runtime code generation → mmap with PROT_EXEC, write machine code
• Stack-to-register mapping → Converting stack operations to register operations
• Deoptimization → Falling back to interpreter when assumptions break
• Code cache management → Limiting memory, invalidating stale code

Key Concepts:

• JIT Compilation: “Language Implementation Patterns” Chapter 11 - Terence Parr
• Executable Memory: “The Linux Programming Interface” Chapter 49 (mmap) - Michael Kerrisk
• Trace Compilation: LuaJIT 2.0 Intellectual Property Disclosure - Mike Pall
• Code Generation: “Engineering a Compiler” Chapter 7 - Cooper & Torczon

Difficulty: Advanced Time estimate: 3-4 weeks Prerequisites: Projects 1-3, understanding of stack machines

Real world outcome:

$ ./jitvm mandelbrot.bc
[JIT] Detected hot loop at bytecode offset 0x42
[JIT] Compiling 23 bytecode ops to 156 bytes of x86-64
[JIT] Code cached at 0x7f4a12340000

████████████████████████████████████████
██████████████████  ████████████████████
████████████████      ██████████████████
██████████████          ████████████████
████████████    ████      ██████████████
██████████  ██████████      ████████████
████████  ████████████        ██████████
████████  ██████████████        ████████

Execution time: 0.089s (JIT enabled)
Execution time: 4.231s (interpreter only)

You’ll see your JIT detect hot loops and compile them to blazing-fast native code.

Implementation Hints: Your bytecode VM has a simple instruction set:

• PUSH n, POP, DUP
• ADD, SUB, MUL, DIV
• LOAD addr, STORE addr
• JMP offset, JZ offset, JNZ offset
• CALL, RET

The interpreter tracks execution counts. When a backward branch (loop) exceeds threshold (e.g., 1000), trigger JIT compilation.

For JIT:

void* code = mmap(NULL, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
                   MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
// Emit x86-64 bytes to code buffer
// Update dispatch table to point to JIT code

Key insight: The JIT is an optimization, not a replacement. Fall back to interpreter for cold code and edge cases.

Learning milestones:

1. mmap+PROT_EXEC works → You can execute generated code
2. Simple arithmetic JITs correctly → Basic code generation works
3. Loops JIT correctly → You’ve handled control flow
4. 10x speedup on hot loops → Your JIT is effective
5. Interpreter/JIT switching is seamless → Full integration

---

Project 5: Shadow Page Table Simulator

• File: HYPERVISOR_VIRTUALIZATION_DEEP_DIVE_PROJECTS.md
• Main Programming Language: C
• Alternative Programming Languages: Rust, C++
• Coolness Level: Level 4: Hardcore Tech Flex
• Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
• Difficulty: Level 3: Advanced (The Engineer)
• Knowledge Area: Memory Virtualization / Page Tables
• Software or Tool: Memory Simulator
• Main Book: “Operating Systems: Three Easy Pieces” by Arpaci-Dusseau

What you’ll build: A software simulation of shadow page tables—the pre-EPT technique for memory virtualization where the VMM maintains a “shadow” of the guest’s page tables.

Why it teaches virtualization: Memory virtualization is half of what makes a hypervisor work. Before EPT/NPT hardware, hypervisors had to trap every guest page table modification and update shadow tables. Understanding this reveals why hardware-assisted memory virtualization was such a breakthrough.

Core challenges you’ll face:

• Three address spaces → Guest virtual, guest physical, host physical
• Write-protecting guest page tables → Detecting when guest modifies its tables
• Shadow table synchronization → Keeping shadow tables consistent with guest tables
• TLB simulation → Understanding when translations are cached
• Performance analysis → Counting VMEXITs to understand overhead

Key Concepts:

• Page Tables: “Operating Systems: Three Easy Pieces” Chapter 18-20 - Arpaci-Dusseau
• Shadow Page Tables: MMU Virtualization: Shadow Page Tables - Ryan Stan
• Memory Virtualization Overview: CSE Washington Lecture
• Address Translation: “Computer Systems: A Programmer’s Perspective” Chapter 9 - Bryant & O’Hallaron

Difficulty: Advanced Time estimate: 2-3 weeks Prerequisites: Understanding of paging and virtual memory

Real world outcome:

$ ./shadow_sim guest_program.bin

=== Memory Virtualization Simulator ===
Guest Physical Memory: 64MB
Host Physical Memory: 256MB

[GUEST] Allocates page at GVA 0x1000 → GPA 0x5000
[VMM] Guest wrote to page table! Trapping...
[VMM] Updating shadow: GVA 0x1000 → HPA 0x8A000
[VMM] Write-protecting guest PTE at GPA 0x2004

[GUEST] Accesses GVA 0x1000
[CPU] TLB miss, walking shadow table
[CPU] Shadow PTE: GVA 0x1000 → HPA 0x8A000
[CPU] Access completed, TLB filled

=== Statistics ===
Guest page table modifications: 1,247
Shadow table updates: 1,247
TLB flushes: 89
Estimated VM exits (real hardware): 1,336

You’ll see exactly why shadow page tables cause so many VM exits and understand the performance impact.

Implementation Hints: Model the address translation as three layers:

1. Guest Page Table (controlled by guest OS) - maps GVA → GPA
2. Shadow Page Table (maintained by VMM) - maps GVA → HPA directly
3. Physical Memory Map (VMM allocation) - maps GPA → HPA

When guest tries to write its page table:

1. Trap the write (because you marked guest PTs read-only)
2. Apply the write to guest’s page table
3. Compute corresponding HPA from your GPA→HPA map
4. Update shadow page table with GVA→HPA mapping
5. Resume guest

Key insight: Shadow tables eliminate one level of indirection at runtime (GVA→HPA directly) but require trapping every page table modification.

Learning milestones:

1. Single-level paging simulated → Basic address translation works
2. Shadow table updates on PT writes → You understand the trap-and-update mechanism
3. Multi-level paging (4-level x86-64) → Full page table hierarchy
4. Statistics show VM exit cost → You understand the performance problem
5. Compare to EPT simulation → You see why hardware helps

---

Project 6: User-Space Memory Mapper with Protection

• File: HYPERVISOR_VIRTUALIZATION_DEEP_DIVE_PROJECTS.md
• Main Programming Language: C
• Alternative Programming Languages: Rust
• Coolness Level: Level 3: Genuinely Clever
• Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
• Difficulty: Level 2: Intermediate (The Developer)
• Knowledge Area: Memory Management / mmap
• Software or Tool: Memory Manager
• Main Book: “The Linux Programming Interface” by Michael Kerrisk

What you’ll build: A user-space memory allocator that simulates guest physical memory using mmap, with support for memory-mapped I/O regions and access permission enforcement.

Why it teaches virtualization: Every hypervisor needs to manage guest memory. This project teaches you how QEMU allocates and manages guest RAM, how MMIO regions work, and how to use Linux memory APIs that are fundamental to any VM implementation.

Core challenges you’ll face:

• Large contiguous allocations → mmap for guest RAM
• MMIO region registration → Carving out address ranges for devices
• Access permission handling → SIGSEGV handlers for MMIO traps
• Memory hot-plug simulation → Adding/removing memory at runtime
• Dirty page tracking → Which pages were modified (for live migration)

Key Concepts:

• mmap and Memory Mapping: “The Linux Programming Interface” Chapter 49 - Michael Kerrisk
• Signal Handling: “The Linux Programming Interface” Chapter 20-22 - Michael Kerrisk
• Guest Memory Management: QEMU Memory API - QEMU Documentation
• Virtual Memory: “Computer Systems: A Programmer’s Perspective” Chapter 9 - Bryant & O’Hallaron

Difficulty: Intermediate Time estimate: 1-2 weeks Prerequisites: Linux system programming basics

Real world outcome:

$ ./memmap_demo

[MEMMAP] Allocating 512MB guest RAM at 0x7f0000000000
[MEMMAP] Registered MMIO region: 0x7f0010000000-0x7f0010001000 (UART)
[MEMMAP] Registered MMIO region: 0x7f0010001000-0x7f0010002000 (VGA)

[GUEST] Writing to RAM at offset 0x1000: OK
[GUEST] Writing to UART at offset 0x10000000: TRAPPED!
  → UART handler called with value 0x41 ('A')
  → Character 'A' printed to console

[MEMMAP] Scanning dirty pages...
  → 47 pages modified since last scan

[MEMMAP] Simulating hot-plug: adding 256MB
[MEMMAP] New guest RAM size: 768MB

You’ll have built the memory management layer that QEMU and similar VMMs use.

Implementation Hints:

// Allocate guest RAM
void* guest_ram = mmap(NULL, 512*1024*1024,
                        PROT_READ|PROT_WRITE,
                        MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);

// Register MMIO region (no permission initially)
void* mmio = mmap((void*)0x7f0010000000, 4096,
                   PROT_NONE,  // No access - will trap
                   MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED, -1, 0);

// Handle access violation
void sigsegv_handler(int sig, siginfo_t *info, void *context) {
    void* addr = info->si_addr;
    if (is_mmio_region(addr)) {
        handle_mmio_access(addr, context);
        return;  // Resume execution
    }
    // Real fault - abort
}

For dirty tracking, use mprotect() to mark pages read-only, then track writes via SIGSEGV.

Learning milestones:

1. mmap allocates guest RAM → Basic memory management works
2. MMIO regions trap correctly → Signal handling works
3. MMIO handlers execute → You can emulate device access
4. Dirty tracking works → You understand live migration foundations
5. Hot-plug works → Dynamic memory management

---

Project 7: Virtual Serial Port (UART) Emulator

• File: HYPERVISOR_VIRTUALIZATION_DEEP_DIVE_PROJECTS.md
• Main Programming Language: C
• Alternative Programming Languages: Rust, C++
• Coolness Level: Level 3: Genuinely Clever
• Business Potential: 3. The “Service & Support” Model (B2B Utility)
• Difficulty: Level 2: Intermediate (The Developer)
• Knowledge Area: Device Emulation / UART
• Software or Tool: Virtual UART
• Main Book: “Linux Device Drivers” by Corbet, Rubini & Kroah-Hartman

What you’ll build: A complete emulation of the 16550 UART (the standard PC serial port) that can connect a VM’s serial output to a host PTY or socket.

Why it teaches virtualization: The UART is the simplest real device with complex behavior: registers, FIFOs, interrupts. Every kernel’s first debug output goes to serial. Understanding UART emulation teaches you the device emulation pattern used by QEMU for hundreds of devices.

Core challenges you’ll face:

• Register-level emulation → THR, RBR, IER, IIR, FCR, LCR, MCR, LSR, MSR
• FIFO management → 16-byte receive/transmit buffers
• Interrupt generation → When to raise IRQ (data ready, transmit empty, etc.)
• Baud rate simulation → Optional: simulate transmission delays
• Backend connection → Connect to host PTY, socket, or file

Key Concepts:

• 16550 UART Specification: 16550 UART Datasheet - Texas Instruments
• Device Registers: “Linux Device Drivers” Chapter 9 - Corbet, Rubini & Kroah-Hartman
• Interrupt Handling: “Linux Device Drivers” Chapter 10 - Corbet, Rubini & Kroah-Hartman
• QEMU Device Model: Understanding QEMU Devices - QEMU Blog

Difficulty: Intermediate Time estimate: 1-2 weeks Prerequisites: Project 6, understanding of hardware registers

Real world outcome:

$ ./uart_emulator --backend=pty

Virtual UART initialized
Backend: /dev/pts/5
Connect with: screen /dev/pts/5 115200

# In another terminal:
$ screen /dev/pts/5 115200

# In your guest OS:
[GUEST] echo "Hello from VM" > /dev/ttyS0

# In screen:
Hello from VM

Your emulated UART can be used by a real operating system for serial I/O!

Implementation Hints: The UART has 8 registers at consecutive I/O ports:

struct uart_state {
    uint8_t thr;      // 0: Transmit Hold (write)
    uint8_t rbr;      // 0: Receive Buffer (read)
    uint8_t ier;      // 1: Interrupt Enable
    uint8_t iir;      // 2: Interrupt ID (read)
    uint8_t fcr;      // 2: FIFO Control (write)
    uint8_t lcr;      // 3: Line Control
    uint8_t mcr;      // 4: Modem Control
    uint8_t lsr;      // 5: Line Status
    uint8_t msr;      // 6: Modem Status
    uint8_t scr;      // 7: Scratch

    uint8_t rx_fifo[16];
    uint8_t tx_fifo[16];
    int rx_count, tx_count;
    int irq_pending;
};

Key behavior:

• Write to THR → add byte to TX FIFO, update LSR
• Read from RBR → remove byte from RX FIFO, update LSR, clear interrupt
• LSR bit 0 = data ready, bit 5 = THR empty

Learning milestones:

1. Register reads/writes work → Basic MMIO emulation
2. TX to PTY works → Guest can send data
3. RX from PTY works → Guest can receive data
4. Interrupts work → Guest IRQ handler receives events
5. Linux kernel boots with your UART → Full compatibility

---

Project 8: Virtual Block Device (Disk) Emulator

• File: HYPERVISOR_VIRTUALIZATION_DEEP_DIVE_PROJECTS.md
• Main Programming Language: C
• Alternative Programming Languages: Rust, Go
• Coolness Level: Level 3: Genuinely Clever
• Business Potential: 3. The “Service & Support” Model (B2B Utility)
• Difficulty: Level 3: Advanced (The Engineer)
• Knowledge Area: Block Device Emulation / Storage
• Software or Tool: Virtual Disk
• Main Book: “Understanding the Linux Kernel” by Bovet & Cesati

What you’ll build: A virtio-blk compatible virtual disk that uses a file or raw device as backing storage, supporting read, write, and flush operations.

Why it teaches virtualization: Storage is critical for VMs. You’ll learn virtio—the standard paravirtualized device interface used by QEMU/KVM. Unlike emulating legacy IDE (slow, many register accesses), virtio uses shared memory queues for efficiency.

Core challenges you’ll face:

• Virtio queue protocol → Available ring, used ring, descriptor chains
• Asynchronous I/O → Non-blocking disk operations with io_uring or AIO
• Request parsing → Understanding virtio-blk request format
• Backing store management → Raw files, qcow2 (optional), sparse files
• Write barriers/flush → Ensuring data durability

Key Concepts:

• Virtio Specification: Virtual I/O Device (VIRTIO) Spec - OASIS
• Block Device Basics: “Understanding the Linux Kernel” Chapter 14 - Bovet & Cesati
• Asynchronous I/O: “The Linux Programming Interface” Chapter 63 - Michael Kerrisk
• QEMU Block Layer: QEMU Block Layer Documentation

Difficulty: Advanced Time estimate: 3-4 weeks Prerequisites: Projects 6-7, understanding of block devices

Real world outcome:

$ dd if=/dev/zero of=disk.img bs=1M count=512
$ mkfs.ext4 disk.img
$ ./vblk_emulator disk.img

[VBLK] Virtio-blk device initialized
[VBLK] Backing store: disk.img (512MB)
[VBLK] Virtqueue at GPA 0x10000, 256 entries

# In guest Linux:
$ lsblk
NAME    SIZE
vda     512M

$ mount /dev/vda /mnt
$ echo "Hello" > /mnt/test.txt
$ sync

# Back in emulator:
[VBLK] READ  sector 2048, count 8
[VBLK] WRITE sector 4096, count 16
[VBLK] FLUSH

Your virtual disk can be formatted, mounted, and used by a real Linux guest!

Implementation Hints: Virtio uses a split-ring structure:

struct vring_desc {
    uint64_t addr;   // Guest physical address
    uint32_t len;    // Length
    uint16_t flags;  // NEXT, WRITE, INDIRECT
    uint16_t next;   // Next descriptor index
};

struct vring_avail {
    uint16_t flags;
    uint16_t idx;          // Next entry guest will write
    uint16_t ring[];       // Descriptor indices
};

struct vring_used {
    uint16_t flags;
    uint16_t idx;          // Next entry host will write
    struct {
        uint32_t id;       // Descriptor index
        uint32_t len;      // Bytes written
    } ring[];
};

Processing loop:

1. Check if avail->idx > last_seen_avail
2. Get descriptor chain from avail->ring[last_seen_avail % size]
3. Parse virtio-blk request header (type, sector, ioprio)
4. Perform I/O on backing file
5. Add to used ring, update used->idx
6. Inject interrupt

Learning milestones:

1. Virtqueue parsing works → You understand virtio protocol
2. Reads work → Guest can read from your virtual disk
3. Writes work → Guest can write to your virtual disk
4. Flush works → Data durability is correct
5. Linux boots from your disk → Full compatibility

---

Project 9: Virtual Network Interface (virtio-net)

• File: HYPERVISOR_VIRTUALIZATION_DEEP_DIVE_PROJECTS.md
• Main Programming Language: C
• Alternative Programming Languages: Rust, Go
• Coolness Level: Level 4: Hardcore Tech Flex
• Business Potential: 3. The “Service & Support” Model (B2B Utility)
• Difficulty: Level 3: Advanced (The Engineer)
• Knowledge Area: Network Virtualization / TAP Devices
• Software or Tool: Virtual NIC
• Main Book: “Understanding Linux Network Internals” by Christian Benvenuti

What you’ll build: A virtio-net compatible virtual network interface card that connects to a host TAP device, enabling the guest to access the network.

Why it teaches virtualization: Network virtualization is where things get real. Your VM can talk to the internet! You’ll learn how packets flow between guest and host, how TAP devices work, and the virtio-net protocol.

Core challenges you’ll face:

• TAP device setup → Creating and configuring host TAP interface
• Virtio-net queues → Separate RX and TX queues with different formats
• Packet header handling → virtio-net header, checksums, GSO
• MAC address management → Assigning and handling MAC addresses
• Multiqueue support → Optional: multiple TX/RX queues for performance

Key Concepts:

• Virtio-net Spec: VIRTIO Spec - Network Device - OASIS
• TAP Devices: “Understanding Linux Network Internals” Chapter 14 - Christian Benvenuti
• Packet Structure: “TCP/IP Illustrated, Volume 1” Chapter 3 - W. Richard Stevens
• QEMU Net: QEMU Networking Documentation

Difficulty: Advanced Time estimate: 3-4 weeks Prerequisites: Project 8, networking basics

Real world outcome:

# Create TAP device
$ sudo ip tuntap add dev tap0 mode tap
$ sudo ip addr add 192.168.100.1/24 dev tap0
$ sudo ip link set tap0 up

$ ./vnet_emulator --tap=tap0

[VNET] Virtio-net device initialized
[VNET] MAC: 52:54:00:12:34:56
[VNET] Connected to tap0

# In guest Linux:
$ ip addr add 192.168.100.2/24 dev eth0
$ ip link set eth0 up
$ ping 192.168.100.1
PING 192.168.100.1: 64 bytes from 192.168.100.1: seq=0 ttl=64 time=0.5ms

# Back in emulator:
[VNET] TX: 98 bytes (ICMP echo request)
[VNET] RX: 98 bytes (ICMP echo reply)

Your guest can ping the host! With NAT or bridging, it can access the internet.

Implementation Hints: virtio-net uses two queues: receiveq (host→guest) and transmitq (guest→host).

Each packet has a header:

struct virtio_net_hdr {
    uint8_t flags;        // NEEDS_CSUM, etc.
    uint8_t gso_type;     // NONE, TCPV4, UDP, etc.
    uint16_t hdr_len;     // Ethernet + IP + TCP header length
    uint16_t gso_size;    // Bytes in each GSO segment
    uint16_t csum_start;  // Position to start checksum
    uint16_t csum_offset; // Offset to place checksum
};

To receive from TAP:

int len = read(tap_fd, buffer, sizeof(buffer));
// Prepend virtio_net_hdr
// Add to guest RX virtqueue
// Inject interrupt

To transmit to TAP:

// Get buffer from guest TX virtqueue
// Strip virtio_net_hdr
write(tap_fd, packet_data, packet_len);

Learning milestones:

1. TAP device connected → Host-side networking ready
2. TX works → Guest can send packets
3. RX works → Guest can receive packets
4. Ping works → ICMP end-to-end
5. TCP works (wget/curl) → Full network stack

---

Project 10: VMX Capability Explorer

• File: HYPERVISOR_VIRTUALIZATION_DEEP_DIVE_PROJECTS.md
• Main Programming Language: C
• Alternative Programming Languages: Rust
• Coolness Level: Level 4: Hardcore Tech Flex
• Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
• Difficulty: Level 3: Advanced (The Engineer)
• Knowledge Area: Intel VT-x / Hardware Virtualization
• Software or Tool: VT-x Explorer
• Main Book: “Intel SDM Volume 3C” (Chapter 23-33)

What you’ll build: A Linux kernel module that queries and displays all Intel VT-x capabilities: supported controls, MSRs, EPT capabilities, and VMCS fields.

Why it teaches virtualization: Before you can build a hypervisor, you need to understand what the hardware offers. This project makes you read the Intel SDM (the bible of x86) and understand CPUID, MSRs, and VMX capabilities.

Core challenges you’ll face:

• CPUID enumeration → Checking VMX support (CPUID.1:ECX.VMX)
• MSR reading → IA32_VMX_BASIC, IA32_VMX_PINBASED_CTLS, etc.
• Capability interpretation → Parsing bitmasks into human-readable features
• Kernel module development → Writing, loading, debugging kernel code
• Feature matrix generation → What your CPU supports vs. requires

Key Concepts:

• VMX Capability MSRs: Intel SDM Vol. 3C Appendix A - Intel
• CPUID Instruction: Intel SDM Vol. 2A - CPUID Reference - Intel
• Kernel Module Development: “Linux Device Drivers” Chapter 2 - Corbet, Rubini & Kroah-Hartman
• VMX Overview: Basic concepts in Intel VT-x - HyperDbg

Difficulty: Advanced Time estimate: 1-2 weeks Prerequisites: Linux kernel module basics, ability to read Intel SDM

Real world outcome:

$ sudo insmod vmx_explorer.ko
$ dmesg | tail -50

[VMX_EXPLORER] Intel VT-x Capability Report
[VMX_EXPLORER] ================================

[VMX_EXPLORER] CPUID.1:ECX.VMX = 1 (VMX supported)
[VMX_EXPLORER] IA32_FEATURE_CONTROL.LOCK = 1

[VMX_EXPLORER] IA32_VMX_BASIC:
  VMCS revision ID: 0x12
  VMCS region size: 4096 bytes
  Memory type: Write-back
  INS/OUTS VMEXIT info: Supported

[VMX_EXPLORER] Pin-Based VM-Execution Controls:
  External interrupt exiting: Allowed
  NMI exiting: Allowed
  Virtual NMIs: Allowed
  VMX preemption timer: Allowed

[VMX_EXPLORER] Primary Processor-Based Controls:
  HLT exiting: Allowed
  INVLPG exiting: Allowed
  MWAIT exiting: Allowed
  RDPMC exiting: Allowed
  RDTSC exiting: Allowed
  CR3-load exiting: Allowed
  CR3-store exiting: Allowed
  CR8-load exiting: Allowed
  CR8-store exiting: Allowed
  Use TPR shadow: Allowed
  NMI-window exiting: Allowed
  MOV-DR exiting: Allowed
  Unconditional I/O exiting: Allowed
  Use I/O bitmaps: Allowed
  Monitor trap flag: Allowed
  Use MSR bitmaps: Allowed
  MONITOR exiting: Allowed
  PAUSE exiting: Allowed
  Activate secondary controls: Allowed

[VMX_EXPLORER] Secondary Processor-Based Controls:
  Enable EPT: Allowed
  Enable VPID: Allowed
  Unrestricted guest: Allowed

[VMX_EXPLORER] EPT Capabilities:
  Execute-only pages: Supported
  Page-walk length 4: Supported
  4KB pages: Supported
  2MB pages: Supported
  1GB pages: Supported

[VMX_EXPLORER] Your CPU supports all requirements for a modern hypervisor!

You’ll know exactly what your CPU can do for virtualization.

Implementation Hints: Key MSRs to read:

#define IA32_VMX_BASIC              0x480
#define IA32_VMX_PINBASED_CTLS      0x481
#define IA32_VMX_PROCBASED_CTLS     0x482
#define IA32_VMX_EXIT_CTLS          0x483
#define IA32_VMX_ENTRY_CTLS         0x484
#define IA32_VMX_PROCBASED_CTLS2    0x48B
#define IA32_VMX_EPT_VPID_CAP       0x48C

uint64_t read_msr(uint32_t msr) {
    uint32_t low, high;
    asm volatile("rdmsr" : "=a"(low), "=d"(high) : "c"(msr));
    return ((uint64_t)high << 32) | low;
}

Each control MSR has:

• Bits 31:0 = “allowed 0-settings” (must be 0 if bit is 0)
• Bits 63:32 = “allowed 1-settings” (can be 1 if bit is 1)

A control is “allowed” if you can set it to either 0 or 1 (bit is 1 in high word).

Learning milestones:

1. Module loads and reads CPUID → Basic kernel module works
2. MSRs read correctly → You can access VMX capabilities
3. All capability MSRs decoded → Comprehensive feature report
4. EPT/VPID capabilities shown → You understand memory virtualization hardware
5. Compare to KVM’s checks → Validate against real hypervisor

---

Project 11: Minimal VT-x Hypervisor (Type 2)

• File: HYPERVISOR_VIRTUALIZATION_DEEP_DIVE_PROJECTS.md
• Main Programming Language: C
• Alternative Programming Languages: Rust
• Coolness Level: Level 5: Pure Magic (Super Cool)
• Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
• Difficulty: Level 4: Expert (The Systems Architect)
• Knowledge Area: Hypervisor Development / VT-x
• Software or Tool: Type-2 Hypervisor
• Main Book: “Intel SDM Volume 3C” (Chapter 23-33)

What you’ll build: A minimal hypervisor that uses Intel VT-x to run a simple guest (real-mode code that prints to VGA) with proper VMCS setup and VM exit handling.

Why it teaches virtualization: This is the core of what KVM, Xen, VMware do. You’ll set up VMCS, enter VMX operation, launch a guest, and handle VM exits. After this, hypervisors won’t be magic anymore.

Core challenges you’ll face:

• VMXON setup → Allocating VMXON region, entering VMX root mode
• VMCS configuration → All the control fields, guest/host state
• Guest state initialization → Setting up the guest CPU state
• VM exit handling → Determining exit reason, handling appropriately
• VM entry/resume → VMLAUNCH for first entry, VMRESUME thereafter

Key Concepts:

• VMCS Structure: Intel SDM Vol. 3C Chapter 24 - Intel
• VM Entries/Exits: Intel SDM Vol. 3C Chapter 25-27 - Intel
• Hypervisor From Scratch: Part 1-8 Tutorial Series - Sina Karvandi
• 1000 Lines Hypervisor: Writing a Hypervisor in 1000 Lines - Seiya Nuta

Difficulty: Expert Time estimate: 4-6 weeks Prerequisites: Projects 10, deep understanding of x86 architecture

Real world outcome:

$ sudo insmod myhypervisor.ko
$ sudo ./launch_guest guest.bin

[HYPERVISOR] Entering VMX operation...
[HYPERVISOR] VMXON successful at 0xffff888012340000
[HYPERVISOR] VMCS allocated at 0xffff888012341000
[HYPERVISOR] Guest state configured (real mode)
[HYPERVISOR] Launching guest with VMLAUNCH...

[GUEST OUTPUT]
Hello from the guest!
This is running in VMX non-root mode!

[HYPERVISOR] VM EXIT #1: Reason = HLT (12)
[HYPERVISOR] Guest executed HLT, shutting down
[HYPERVISOR] Guest RIP was 0x7c42
[HYPERVISOR] Total instructions executed in guest: ~847
[HYPERVISOR] Exiting VMX operation...

$ dmesg | tail
[HYPERVISOR] VMX operation exited cleanly

Your code ran a guest in hardware virtualization! The CPU was literally in a different execution mode.

Implementation Hints: High-level flow:

// 1. Enable VMX
set_cr4_vmxe();

// 2. Enter VMX operation
vmxon_region = alloc_vmxon_region();
vmxon(vmxon_region);

// 3. Create and configure VMCS
vmcs = alloc_vmcs();
vmclear(vmcs);
vmptrld(vmcs);

// 4. Write VMCS fields
vmwrite(GUEST_CS_SELECTOR, 0);
vmwrite(GUEST_CS_BASE, 0);
vmwrite(GUEST_RIP, 0x7c00);  // Boot sector start
vmwrite(GUEST_RSP, 0x7000);
vmwrite(GUEST_RFLAGS, 0x2);
// ... many more fields

// 5. Launch guest
vmlaunch();

// 6. Handle VM exit (we get here on exit)
uint32_t reason = vmread(VM_EXIT_REASON);
handle_exit(reason);

// 7. Resume or terminate
vmresume();  // or vmxoff()

The VMCS has ~100 fields. Start with the minimum:

• Guest state: CS/SS/DS/ES/FS/GS selectors and bases, CR0/CR3/CR4, RIP, RSP, RFLAGS
• Host state: Same fields for returning after VM exit
• Control fields: What causes VM exits

Learning milestones:

1. VMXON succeeds → You’re in VMX root operation
2. VMCS configured without errors → All mandatory fields set correctly
3. VMLAUNCH succeeds → Guest code executes
4. VM exit captured → You can see why guest exited
5. Multiple VM entries/exits → Guest can do useful work

---

Project 12: EPT (Extended Page Tables) Implementation

• File: HYPERVISOR_VIRTUALIZATION_DEEP_DIVE_PROJECTS.md
• Main Programming Language: C
• Alternative Programming Languages: Rust
• Coolness Level: Level 5: Pure Magic (Super Cool)
• Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
• Difficulty: Level 4: Expert (The Systems Architect)
• Knowledge Area: Memory Virtualization / EPT
• Software or Tool: EPT Implementation
• Main Book: “Intel SDM Volume 3C” (Chapter 28)

What you’ll build: Extend your hypervisor with EPT support, allowing the guest to have its own physical address space that maps to real host physical memory.

Why it teaches virtualization: EPT is how modern hypervisors do memory virtualization. Without it, you’d need shadow page tables (slow). With EPT, the hardware translates guest-physical to host-physical automatically.

Core challenges you’ll face:

• EPT structure setup → 4-level page tables for GPA→HPA translation
• EPTP configuration → Pointing VMCS to your EPT root
• EPT violation handling → What happens when guest accesses unmapped memory
• Identity mapping → Simple 1:1 GPA→HPA initially
• MMIO carve-out → Marking MMIO regions as not present in EPT

Key Concepts:

• EPT Mechanism: Intel SDM Vol. 3C Chapter 28.2 - Intel
• EPT Paging Structures: Intel SDM Vol. 3C Chapter 28.2.2 - Intel
• EPT Tutorial: Hypervisor From Scratch Part 4 - EPT - Sina Karvandi
• Memory Virtualization Overview: UCSD Memory Virtualization Slides

Difficulty: Expert Time estimate: 2-3 weeks Prerequisites: Project 11, understanding of paging

Real world outcome:

$ sudo ./hypervisor_with_ept guest.bin

[HYPERVISOR] Building EPT structures...
[HYPERVISOR] EPT PML4 at HPA 0x100000000
[HYPERVISOR] Mapping guest memory:
  GPA 0x00000000-0x00100000 → HPA 0x200000000 (1MB, RWX)
  GPA 0x00100000-0x10000000 → HPA 0x200100000 (255MB, RWX)
[HYPERVISOR] MMIO regions (not in EPT):
  GPA 0xB8000-0xB9000 (VGA)

[HYPERVISOR] EPT enabled in VMCS
[HYPERVISOR] Launching guest...

[GUEST OUTPUT]
Testing memory at 0x100000... OK
Testing memory at 0x1000000... OK
Writing to VGA at 0xB8000...

[HYPERVISOR] EPT VIOLATION at GPA 0xB8000 (VGA write)
[HYPERVISOR] Emulating VGA write: char='H' at row=0, col=0
[HYPERVISOR] Resuming guest...

[GUEST OUTPUT]
Hello World! (via emulated VGA)

The guest thinks it has its own physical memory! Hardware does the translation.

Implementation Hints: EPT has 4 levels like regular x86-64 paging:

// EPT entry format (same for all levels)
struct ept_entry {
    uint64_t read       : 1;   // Allow reads
    uint64_t write      : 1;   // Allow writes
    uint64_t execute    : 1;   // Allow execute
    uint64_t mem_type   : 3;   // Memory type (0=UC, 6=WB)
    uint64_t ignore_pat : 1;
    uint64_t large_page : 1;   // 2MB/1GB page
    uint64_t accessed   : 1;
    uint64_t dirty      : 1;
    uint64_t reserved   : 2;
    uint64_t phys_addr  : 40;  // Physical address of next level/page
    uint64_t reserved2  : 12;
};

Build identity map for simplicity:

// Map 0-512GB identity (GPA = HPA)
for (i = 0; i < 512; i++) {
    ept_pml4[i] = make_ept_entry(ept_pdpt[i], R|W|X);
    for (j = 0; j < 512; j++) {
        // Use 1GB pages for simplicity
        ept_pdpt[i][j] = make_ept_entry((i*512 + j) * 1GB, R|W|X|LARGE);
    }
}

For MMIO, leave those GPAs unmapped → EPT violation → emulate in hypervisor.

Learning milestones:

1. EPT structures built correctly → Valid page tables
2. Guest can access RAM → EPT translates correctly
3. EPT violations on MMIO → You can intercept device access
4. VGA writes work → Guest can display output through EPT traps
5. Compare performance to shadow PT → Measure improvement

---

Project 13: Mini-QEMU Clone

• File: HYPERVISOR_VIRTUALIZATION_DEEP_DIVE_PROJECTS.md
• Main Programming Language: C
• Alternative Programming Languages: Rust
• Coolness Level: Level 5: Pure Magic (Super Cool)
• Business Potential: 2. The “Micro-SaaS / Pro Tool” (Solo-Preneur Potential)
• Difficulty: Level 4: Expert (The Systems Architect)
• Knowledge Area: Full System Emulation
• Software or Tool: System Emulator
• Main Book: Multiple (see resources)

What you’ll build: A user-space emulator like QEMU that combines a CPU emulator (from Projects 2-4), memory manager (Project 6), and devices (Projects 7-9) to run a simple OS.

Why it teaches virtualization: This integrates everything. You’ll understand how QEMU works internally: the main loop, device dispatch, interrupt injection, and how all pieces fit together.

Core challenges you’ll face:

• Component integration → CPU, memory, devices in one system
• Device dispatch → Routing MMIO to correct device
• Interrupt controller → Emulating PIC or APIC
• Main loop design → When to run CPU vs. check devices
• Boot process → Loading and executing BIOS/bootloader

Key Concepts:

• QEMU Internals: Airbus QEMU Internals Blog - Airbus Security Lab
• System Organization: “Computer Systems: A Programmer’s Perspective” Chapter 1 - Bryant & O’Hallaron
• PC Architecture: QEMU PC System Emulation - QEMU Docs
• BIOS/Boot: “How Computers Really Work” Chapter 10 - Matthew Justice

Difficulty: Expert Time estimate: 6-8 weeks Prerequisites: Projects 2, 6, 7, 8, 9

Real world outcome:

$ ./miniemu --ram=64M --disk=freedos.img --serial=stdio

[MINIEMU] Mini-QEMU Clone v0.1
[MINIEMU] CPU: Software RISC-V emulator (RV32I)
[MINIEMU] RAM: 64MB at 0x80000000
[MINIEMU] Devices:
  - UART 16550 at 0x10000000
  - VirtIO-blk at 0x10001000
  - Interrupt controller at 0x0C000000

[MINIEMU] Loading bootloader from sector 0...
[MINIEMU] Jumping to 0x80000000...

FreeDOS kernel loading...
FreeDOS 1.3 [build 2043]

C:\> dir
 Volume in drive C is FREEDOS
 Directory of C:\

KERNEL   SYS     45,945 03-01-2022  12:00p
COMMAND  COM     66,945 03-01-2022  12:00p
        2 file(s)    112,890 bytes

C:\> echo Hello from Mini-QEMU!
Hello from Mini-QEMU!

You’ve built a system emulator! An actual OS boots and runs!

Implementation Hints: Main emulation loop:

void emu_run(void) {
    while (running) {
        // 1. Check for pending interrupts
        if (pic_has_interrupt()) {
            cpu_inject_interrupt(pic_get_irq());
        }

        // 2. Execute some instructions
        for (int i = 0; i < 1000; i++) {
            cpu_step();
        }

        // 3. Update devices
        uart_tick();
        virtio_blk_tick();
        timer_tick();

        // 4. Check for I/O
        poll_io();
    }
}

Device dispatch on MMIO:

void mmio_write(uint64_t addr, uint32_t val) {
    if (addr >= UART_BASE && addr < UART_BASE + UART_SIZE)
        uart_write(addr - UART_BASE, val);
    else if (addr >= VBLK_BASE && addr < VBLK_BASE + VBLK_SIZE)
        virtio_blk_write(addr - VBLK_BASE, val);
    else if (addr >= PIC_BASE && addr < PIC_BASE + PIC_SIZE)
        pic_write(addr - PIC_BASE, val);
    else
        printf("Unknown MMIO write to %lx\n", addr);
}

Learning milestones:

1. CPU + memory integrated → Basic execution works
2. UART works → Guest can print output
3. Disk works → Guest can read files
4. Interrupts work → Timer and device IRQs function
5. Simple OS boots → Full system emulation

---

Project 14: KVM Userspace Client

• File: HYPERVISOR_VIRTUALIZATION_DEEP_DIVE_PROJECTS.md
• Main Programming Language: C
• Alternative Programming Languages: Rust, Go
• Coolness Level: Level 5: Pure Magic (Super Cool)
• Business Potential: 3. The “Service & Support” Model (B2B Utility)
• Difficulty: Level 3: Advanced (The Engineer)
• Knowledge Area: KVM API / Hardware Virtualization
• Software or Tool: KVM Client
• Main Book: “The Linux Programming Interface” by Michael Kerrisk (for ioctl patterns)

What you’ll build: A userspace program that uses Linux’s KVM API to run a virtual machine with hardware acceleration, without writing any kernel code.

Why it teaches virtualization: This is how QEMU works with KVM! KVM exposes Intel VT-x through ioctls. You get near-native performance without implementing the hypervisor yourself.

Core challenges you’ll face:

• KVM API → /dev/kvm ioctls for VM and VCPU management
• Memory mapping → Setting up guest memory with KVM_SET_USER_MEMORY_REGION
• VCPU run loop → KVM_RUN and handling exits
• Register access → KVM_GET/SET_REGS, KVM_GET/SET_SREGS
• I/O handling → Responding to KVM_EXIT_IO and KVM_EXIT_MMIO

Key Concepts:

• KVM API: KVM API Documentation - Linux Kernel Docs
• KVM Internals: LWN KVM Article - LWN.net
• ioctl Programming: “The Linux Programming Interface” Chapter 4 - Michael Kerrisk
• x86 Boot Protocol: Linux x86 Boot Protocol - Kernel Docs

Difficulty: Advanced Time estimate: 2-3 weeks Prerequisites: Linux system programming, basic x86 knowledge

Real world outcome:

$ ./kvmclient guest.bin

[KVM] Opening /dev/kvm...
[KVM] KVM API version: 12
[KVM] Creating VM...
[KVM] Setting up memory: 128MB at userspace 0x7f1234000000
[KVM] Creating VCPU...
[KVM] Setting up registers:
  RIP = 0x0000
  RSP = 0xFFFE
  RFLAGS = 0x0002

[KVM] Running VCPU...

[KVM_EXIT_IO] Port 0x3F8, OUT, data='H'
[KVM_EXIT_IO] Port 0x3F8, OUT, data='e'
[KVM_EXIT_IO] Port 0x3F8, OUT, data='l'
[KVM_EXIT_IO] Port 0x3F8, OUT, data='l'
[KVM_EXIT_IO] Port 0x3F8, OUT, data='o'
[KVM_EXIT_HLT] Guest halted

[KVM] Guest output: Hello
[KVM] VCPU ran for 1,247 exits

Your guest code runs at native speed using KVM!

Implementation Hints:

// Open KVM
int kvm = open("/dev/kvm", O_RDWR);

// Create VM
int vmfd = ioctl(kvm, KVM_CREATE_VM, 0);

// Allocate guest memory
void* mem = mmap(NULL, 128*1024*1024, PROT_READ|PROT_WRITE,
                 MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);

// Map into guest
struct kvm_userspace_memory_region region = {
    .slot = 0,
    .guest_phys_addr = 0,
    .memory_size = 128*1024*1024,
    .userspace_addr = (uint64_t)mem,
};
ioctl(vmfd, KVM_SET_USER_MEMORY_REGION, &region);

// Create VCPU
int vcpufd = ioctl(vmfd, KVM_CREATE_VCPU, 0);

// Get run structure
struct kvm_run* run = mmap(..., ioctl(kvm, KVM_GET_VCPU_MMAP_SIZE, 0), ...);

// Run loop
while (1) {
    ioctl(vcpufd, KVM_RUN, 0);

    switch (run->exit_reason) {
    case KVM_EXIT_IO:
        handle_io(run);
        break;
    case KVM_EXIT_HLT:
        return;
    }
}

Learning milestones:

1. KVM_CREATE_VM works → KVM API access works
2. VCPU runs and exits → Guest code executes
3. I/O exits handled → Guest can do serial output
4. Real-mode guest works → x86 compatibility
5. Protected-mode guest works → Full OS capability

---

Project 15: Complete Type-2 Hypervisor (Capstone)

• File: HYPERVISOR_VIRTUALIZATION_DEEP_DIVE_PROJECTS.md
• Main Programming Language: C
• Alternative Programming Languages: Rust
• Coolness Level: Level 5: Pure Magic (Super Cool)
• Business Potential: 5. The “Industry Disruptor” (VC-Backable Platform)
• Difficulty: Level 5: Master (The First-Principles Wizard)
• Knowledge Area: Complete Hypervisor / VT-x + Devices
• Software or Tool: Full Hypervisor
• Main Book: Intel SDM + “The Definitive Guide to the Xen Hypervisor” by David Chisnall

What you’ll build: A complete Type-2 hypervisor that runs Linux as a guest, with virtio devices, SMP support (multiple VCPUs), and a monitor console.

Why it teaches virtualization: This is the culmination of everything. You’ll have built something comparable to a simple QEMU/KVM or early VirtualBox. You’ll understand every layer from VT-x to device emulation.

Core challenges you’ll face:

• SMP support → Multiple VCPUs, inter-processor interrupts
• APIC emulation → Modern interrupt controller
• Full Linux boot → bzImage loading, kernel command line, initrd
• Virtio integration → Block + network for a usable system
• Monitor/debug console → Inspect and control the VM

Key Concepts:

• Everything from Projects 1-14
• SMP Virtualization: Intel SDM Vol. 3C Chapter 29 - Intel
• APIC Emulation: APIC Specification - OSDev Wiki
• Linux Boot Protocol: Kernel Boot Protocol - Kernel Docs
• Hypervisor Architecture: “The Definitive Guide to the Xen Hypervisor” - David Chisnall

Difficulty: Master Time estimate: 3-6 months Prerequisites: All previous projects

Real world outcome:

$ ./myhypervisor \
    --cpus 4 \
    --memory 2G \
    --disk ubuntu.qcow2 \
    --net user \
    --kernel vmlinuz \
    --initrd initrd.img \
    --append "console=ttyS0 root=/dev/vda1"

[HYPERVISOR] My Hypervisor v1.0
[HYPERVISOR] VCPUs: 4 (VT-x + EPT)
[HYPERVISOR] RAM: 2GB
[HYPERVISOR] Devices: virtio-blk, virtio-net, 16550 UART

[HYPERVISOR] Loading kernel (6,234,112 bytes)...
[HYPERVISOR] Loading initrd (12,456,789 bytes)...
[HYPERVISOR] Starting VCPU 0 at 0x100000...
[HYPERVISOR] Starting VCPU 1-3 (waiting for SIPI)...

[    0.000000] Linux version 5.15.0 (gcc 11.2.0)
[    0.000000] Command line: console=ttyS0 root=/dev/vda1
[    0.001234] smpboot: Allowing 4 CPUs
[    0.045678] virtio_blk virtio0: [vda] 41943040 512-byte sectors
[    0.089012] virtio_net virtio1: MAC: 52:54:00:12:34:56

Ubuntu 22.04 LTS myvm ttyS0

myvm login: root
Password:
Welcome to Ubuntu 22.04 LTS

root@myvm:~# cat /proc/cpuinfo | grep processor
processor	: 0
processor	: 1
processor	: 2
processor	: 3

root@myvm:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP>
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP>
    inet 10.0.2.15/24

root@myvm:~# curl http://example.com
<!doctype html>...

You’ve built a hypervisor that runs a full Linux distribution with networking!

Implementation Hints: This is a large project. Build incrementally:

Phase 1: Single VCPU Linux boot (2-3 weeks)

• Use KVM API (Project 14) as your VT-x backend
• Load bzImage properly (protected mode entry)
• Get to kernel panic (no root filesystem)

Phase 2: Add virtio-blk (1-2 weeks)

• Integrate Project 8
• Boot to login prompt

Phase 3: Add virtio-net (1-2 weeks)

• Integrate Project 9
• Network access works

Phase 4: SMP (2-3 weeks)

• Multiple VCPUs
• SIPI handling (start other CPUs)
• IPI support

Phase 5: Polish (2-4 weeks)

• Monitor console (connect via telnet, inspect state)
• Live migration support (optional)
• Performance tuning

Learning milestones:

1. Kernel decompresses → bzImage loading works
2. Kernel boots to panic → Basic emulation works
3. Reaches login prompt → Disk works
4. SSH/network works → Network works
5. Multiple CPUs detected → SMP works
6. Can run for hours → Stable and complete

---

Project Comparison Table

#	Project	Difficulty	Time	Depth	Fun	Hardware Needed
1	CHIP-8 Interpreter	Beginner	1 week	⭐⭐	⭐⭐⭐⭐⭐	None
2	RISC-V Emulator	Intermediate	2-3 weeks	⭐⭐⭐	⭐⭐⭐⭐	None
3	Basic Block Translator	Advanced	3-4 weeks	⭐⭐⭐⭐	⭐⭐⭐⭐	None
4	JIT Compiler	Advanced	3-4 weeks	⭐⭐⭐⭐	⭐⭐⭐⭐⭐	None
5	Shadow PT Simulator	Advanced	2-3 weeks	⭐⭐⭐⭐⭐	⭐⭐⭐	None
6	User-Space Memory Mapper	Intermediate	1-2 weeks	⭐⭐⭐	⭐⭐⭐	None
7	Virtual UART	Intermediate	1-2 weeks	⭐⭐⭐	⭐⭐⭐	None
8	Virtual Block Device	Advanced	3-4 weeks	⭐⭐⭐⭐	⭐⭐⭐⭐	None
9	Virtual Network Interface	Advanced	3-4 weeks	⭐⭐⭐⭐	⭐⭐⭐⭐⭐	None
10	VMX Capability Explorer	Advanced	1-2 weeks	⭐⭐⭐	⭐⭐⭐	Intel CPU with VT-x
11	Minimal VT-x Hypervisor	Expert	4-6 weeks	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	Intel CPU with VT-x
12	EPT Implementation	Expert	2-3 weeks	⭐⭐⭐⭐⭐	⭐⭐⭐⭐	Intel CPU with EPT
13	Mini-QEMU Clone	Expert	6-8 weeks	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	None
14	KVM Userspace Client	Advanced	2-3 weeks	⭐⭐⭐⭐	⭐⭐⭐⭐⭐	Linux with KVM
15	Complete Type-2 Hypervisor	Master	3-6 months	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	Intel VT-x + EPT

---

Recommended Learning Path

If you’re new to systems programming:

Start with: Project 1 (CHIP-8) → Project 2 (RISC-V) → Project 6 (Memory) → Project 7 (UART)

If you want to understand QEMU:

Focus on: Project 2 → Project 3 → Project 4 → Project 13

If you want to understand hardware virtualization:

Focus on: Project 10 → Project 11 → Project 12 → Project 14

If you want to understand memory virtualization:

Focus on: Project 5 → Project 6 → Project 12

The complete path (6-12 months):

1. Month 1: Projects 1-2 (Emulation fundamentals)
2. Month 2: Projects 3-4 (Binary translation)
3. Month 3: Projects 5-6 (Memory management)
4. Month 4: Projects 7-9 (Device emulation)
5. Month 5-6: Projects 10-12 (Hardware virtualization)
6. Month 7-8: Projects 13-14 (Integration)
7. Month 9-12: Project 15 (Capstone)

---

Essential Resources

Books

1. “Computer Systems: A Programmer’s Perspective” by Bryant & O’Hallaron - Foundation for everything
2. “Intel SDM Volume 3C” (Chapters 23-33) - The bible for VT-x
3. “Operating Systems: Three Easy Pieces” by Arpaci-Dusseau - Memory virtualization
4. “The Definitive Guide to the Xen Hypervisor” by David Chisnall - Hypervisor architecture
5. “The Linux Programming Interface” by Michael Kerrisk - System programming

Online Tutorials

1. Hypervisor From Scratch - 8-part tutorial series
2. Writing a Hypervisor in 1000 Lines - Minimal RISC-V hypervisor
3. QEMU Internals - Deep dive into QEMU
4. HyperDbg Documentation - VT-x concepts explained

Specifications

1. Intel SDM - Official Intel documentation
2. VIRTIO Specification - Virtio device standard
3. KVM API - Linux KVM interface

---

Summary

#	Project Name	Main Language
1	CHIP-8 Interpreter Emulator	C
2	Simple RISC CPU Emulator	C
3	Basic Block Binary Translator	C
4	Simple JIT Compiler for Bytecode VM	C
5	Shadow Page Table Simulator	C
6	User-Space Memory Mapper with Protection	C
7	Virtual Serial Port (UART) Emulator	C
8	Virtual Block Device (Disk) Emulator	C
9	Virtual Network Interface (virtio-net)	C
10	VMX Capability Explorer	C
11	Minimal VT-x Hypervisor	C
12	EPT Implementation	C
13	Mini-QEMU Clone	C
14	KVM Userspace Client	C
15	Complete Type-2 Hypervisor (Capstone)	C

---

After completing these projects, you’ll have gone from “I don’t know if QEMU is a hypervisor” to “I built my own hypervisor that runs Linux.” You’ll understand virtualization at every level: software emulation, binary translation, hardware-assisted virtualization, memory virtualization, and device emulation. This is deep systems knowledge that very few developers possess.