OPERATING SYSTEMS FROM FIRST PRINCIPLES

Operating Systems from First Principles

Goal: Understand how operating systems work at the deepest level—from bare metal to user space—and grasp why Unix’s design philosophy has proven so powerful and enduring.

This learning path doesn’t just teach you OS concepts; it helps you understand why operating systems are designed the way they are, why Unix “won,” and what makes its abstractions so elegant that they’ve survived for 50+ years.

The Big Questions We’ll Answer

What is an operating system, really? (It’s smaller than you think)
Why do we need one? (Hardware is chaos; OS brings order)
What are the core abstractions? (Processes, files, memory—that’s almost it)
Why is Unix so elegant? (Everything is a file, small tools, composition)
How is Windows different? (Different trade-offs, different philosophy)
Why did Unix “win”? (Simplicity scales; complexity doesn’t)

The Unix Philosophy: Why It Matters

Before diving into projects, understand this: Unix isn’t just an operating system—it’s a philosophy about how to build complex systems from simple parts.

The Core Principles

1. Everything is a file
   - Keyboard? File. Screen? File. Network socket? File. Process info? File.
   - One interface (read/write) to rule them all
   - Radical simplification that enables composition

2. Small, sharp tools that do one thing well
   - `cat` just concatenates. `grep` just searches. `sort` just sorts.
   - Combine them: `cat log.txt | grep ERROR | sort | uniq -c`
   - Emergent complexity from simple parts

3. Text streams as universal interface
   - Programs don't need to understand each other's internals
   - Just read lines, process, write lines
   - Pipelines become possible

4. Composition over monoliths
   - Don't build one program that does everything
   - Build many programs that work together
   - The shell is the glue

Why This Won

Windows philosophy:     "Build complete applications with rich APIs"
Unix philosophy:        "Build simple tools that compose"

Windows result:         Complex, tightly coupled, hard to automate
Unix result:            Simple, loosely coupled, infinitely scriptable

When the internet arrived:
  - Unix tools composed into web servers, databases, automation
  - The "everything is a file" model extended naturally to networks
  - Small tools meant small attack surface, easier security

When cloud computing arrived:
  - Unix's text-based configuration meant easy automation
  - Containers are just Unix namespaces + cgroups
  - Kubernetes runs on Unix philosophy

Unix won because simplicity scales. Complexity doesn't.

The Core OS Abstractions (There Are Only Three!)

Everything in an operating system reduces to managing three things:

┌─────────────────────────────────────────────────────────────────┐
│                        USER PROGRAMS                            │
├─────────────────────────────────────────────────────────────────┤
│                     SYSTEM CALL INTERFACE                       │
├───────────────────┬───────────────────┬─────────────────────────┤
│     PROCESSES     │      MEMORY       │      FILES/IO           │
│                   │                   │                         │
│  - Creation       │  - Allocation     │  - Everything is a file │
│  - Scheduling     │  - Virtual memory │  - Read/Write/Open/Close│
│  - Communication  │  - Protection     │  - Directories          │
│  - Termination    │  - Sharing        │  - Devices              │
├───────────────────┴───────────────────┴─────────────────────────┤
│                         HARDWARE                                │
│              CPU         Memory         Disk/Devices            │
└─────────────────────────────────────────────────────────────────┘

That’s it. Processes, memory, files. Everything else is implementation detail.

Windows vs Unix: Architectural Differences

                    UNIX                           WINDOWS
                    ────                           ───────
Kernel Model:       Monolithic                     Hybrid (NT kernel)
                    (everything in kernel)         (some services in user space)

File System:        Hierarchical, single root      Drive letters (C:, D:)
                    /home/user/docs                C:\Users\User\Documents

Everything File:    YES - devices are files        NO - devices are objects
                    /dev/sda, /dev/tty             \\.\PhysicalDrive0

Text Config:        /etc/passwd, ~/.bashrc         Registry (binary database)
                    Human readable, scriptable     GUI-oriented, harder to script

Process Model:      fork() + exec()                CreateProcess()
                    Clone parent, then transform   Create fresh process

IPC:                Pipes, sockets, signals        Named pipes, COM, RPC, WMI
                    Simple, file-like              Rich, complex APIs

Permissions:        User/Group/Other (rwx)         ACLs (Access Control Lists)
                    Simple, coarse                 Fine-grained, complex

Shell:              First-class citizen            Afterthought (until PowerShell)
                    sh/bash/zsh                    cmd.exe was primitive

Philosophy:         "Worse is better"              "The right thing"
                    Simple beats complete          Complete beats simple

Why These Differences Matter

Windows chose richness: Complex APIs, GUI-first, binary formats. Great for desktop applications with rich UIs. Harder to automate, script, compose.

Unix chose simplicity: Text formats, CLI-first, everything is a file. Awkward for desktop GUIs. Trivial to automate, script, compose.

The internet, servers, and cloud rewarded Unix’s choices. Automation won.

Part 1: Understanding the Hardware-Software Boundary

Before you can understand an OS, you must understand what the hardware provides and what it doesn’t.

Project 1: Bare Metal “Hello World” (No OS)

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: Assembly (x86 NASM)
Alternative Programming Languages: x86 GAS syntax, ARM Assembly
Coolness Level: Level 5: Pure Magic (Super Cool)
Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
Difficulty: Level 4: Expert (The Systems Architect)
Knowledge Area: Boot Process / Hardware Interface
Software or Tool: QEMU Emulator
Main Book: “Operating Systems: Three Easy Pieces” by Arpaci-Dusseau

What you’ll build: A 512-byte boot sector that, when loaded by the BIOS, prints “Hello, OS!” directly to the screen—with no operating system, no libraries, no runtime. Just your code and the hardware.

Why it teaches OS fundamentals: This strips away every abstraction. You’ll understand that the CPU is just a machine that fetches instructions and executes them. The BIOS loads your code to a specific memory address (0x7C00), jumps to it, and you’re on your own. No printf, no syscalls—just raw hardware access.

Core challenges you’ll face:

Understanding the boot process → maps to how computers start
Writing to video memory directly → maps to memory-mapped I/O
No standard library, no OS services → maps to what the OS actually provides
Real mode vs protected mode → maps to CPU privilege levels
The 512-byte boot sector limit → maps to bootstrapping constraints

Key Concepts:

Boot Process: “Operating Systems: Three Easy Pieces” Chapter 36 - Arpaci-Dusseau
x86 Real Mode: “Write Great Code, Volume 2” Chapter 3 - Randall Hyde
BIOS Interrupts: “The Art of Assembly Language” Chapter 13 - Randall Hyde
Memory-Mapped I/O: “Computer Systems: A Programmer’s Perspective” Chapter 6 - Bryant & O’Hallaron

Difficulty: Expert Time estimate: Weekend (intense) Prerequisites: Basic assembly understanding, willingness to read documentation

Real world outcome:

$ nasm -f bin boot.asm -o boot.bin
$ qemu-system-x86_64 -drive format=raw,file=boot.bin

[QEMU window opens]
[Black screen with white text:]
Hello, OS!
_

[Cursor blinking. That's YOUR code running on bare metal!]

Implementation Hints: The boot sector must:

Be exactly 512 bytes
End with magic bytes 0x55 0xAA (BIOS signature)
Start at 0x7C00 in memory (BIOS loads it there)

Video memory in text mode starts at 0xB8000. Each character is 2 bytes: ASCII code + attribute (color).

; Pseudo-structure (not actual code!)
; Write 'H' in white on black at position 0
mov word [0xB8000], 0x0F48  ; 0x0F = white on black, 0x48 = 'H'

BIOS interrupt int 0x10 can also print characters, but writing to video memory directly shows you what “no OS” really means.

Learning milestones:

Boot sector loads and executes → You understand the boot process
Text appears on screen → You understand memory-mapped I/O
You realize there’s nothing else → You understand what an OS must provide

Project 2: Bootloader that Loads a Kernel

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: Assembly + C
Alternative Programming Languages: Assembly + Rust, Pure Assembly
Coolness Level: Level 5: Pure Magic (Super Cool)
Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
Difficulty: Level 4: Expert (The Systems Architect)
Knowledge Area: Boot Process / Protected Mode
Software or Tool: Custom Bootloader
Main Book: “Operating Systems: Three Easy Pieces” by Arpaci-Dusseau

What you’ll build: A bootloader that loads a larger kernel from disk into memory, switches the CPU from 16-bit real mode to 32-bit protected mode, and jumps to the kernel. This is what GRUB does, but you’ll build it yourself.

Why it teaches OS fundamentals: The 512-byte boot sector limitation forces a two-stage boot. This teaches disk I/O (reading sectors), memory layout, CPU mode switching, and the hand-off from firmware to OS. You’ll understand why bootloaders exist.

Core challenges you’ll face:

Reading from disk using BIOS → maps to block device I/O
Setting up the Global Descriptor Table (GDT) → maps to memory segmentation
Switching to 32-bit protected mode → maps to privilege and protection
Jumping from 16-bit to 32-bit code → maps to ABI transitions
Loading kernel to correct memory address → maps to memory layout planning

Key Concepts:

Protected Mode: “The Art of Assembly Language” Chapter 8 - Randall Hyde
GDT and Segmentation: “Operating Systems: Three Easy Pieces” Chapter 15 - Arpaci-Dusseau
Disk I/O: “Linux Kernel Development” Chapter 13 - Robert Love
Two-Stage Boot: “Operating Systems Design and Implementation” Chapter 2 - Tanenbaum

Difficulty: Expert Time estimate: 1-2 weeks Prerequisites: Project 1, understanding of x86 architecture

Real world outcome:

$ make
nasm -f bin boot.asm -o boot.bin      # Stage 1 bootloader
nasm -f elf32 kernel_entry.asm -o kernel_entry.o
gcc -m32 -ffreestanding -c kernel.c -o kernel.o
ld -m elf_i386 -T linker.ld -o kernel.bin kernel_entry.o kernel.o
cat boot.bin kernel.bin > os.bin

$ qemu-system-x86_64 -drive format=raw,file=os.bin

[Screen shows:]
Bootloader: Loading kernel from disk...
Bootloader: Switching to protected mode...
Bootloader: Jumping to kernel...

=== MyOS Kernel v0.1 ===
Running in 32-bit protected mode
Kernel loaded at 0x1000
Hello from C!

Implementation Hints: Stage 1 (512 bytes): Use BIOS int 0x13 to read sectors from disk. Load stage 2 / kernel to a known address (e.g., 0x1000).

Protected mode switch requires:

Disable interrupts (cli)
Load GDT (lgdt)
Set PE bit in CR0
Far jump to flush pipeline and load CS with protected mode selector

The GDT defines memory segments. For a flat memory model (which you want), set base=0, limit=4GB for both code and data segments.

Learning milestones:

Kernel loads from disk → You understand block device I/O
Protected mode switch works → You understand CPU privilege levels
C code runs in your kernel → You understand the runtime environment

Project 3: Memory Allocator (malloc from Scratch)

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: C
Alternative Programming Languages: Rust, C++
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
Difficulty: Level 3: Advanced (The Engineer)
Knowledge Area: Memory Management / Data Structures
Software or Tool: malloc Implementation
Main Book: “The Linux Programming Interface” by Michael Kerrisk

What you’ll build: Your own implementation of malloc(), free(), and realloc() that manages a heap, handles fragmentation, and coalesces free blocks. Then use it in real programs.

Why it teaches OS fundamentals: Memory allocation is the heart of the memory management abstraction. Understanding how the heap works—block headers, free lists, splitting, coalescing—reveals what the OS does for every program. You’ll also understand why memory bugs (use-after-free, double-free) are dangerous.

Core challenges you’ll face:

Requesting memory from OS → maps to sbrk() or mmap()
Block headers and metadata → maps to memory layout
Free list management → maps to data structures in constrained memory
Splitting and coalescing → maps to fragmentation prevention
Alignment requirements → maps to hardware constraints

Key Concepts:

Dynamic Memory Allocation: “Computer Systems: A Programmer’s Perspective” Chapter 9 - Bryant & O’Hallaron
sbrk and mmap: “The Linux Programming Interface” Chapter 7 - Michael Kerrisk
Allocator Strategies: “Operating Systems: Three Easy Pieces” Chapter 17 - Arpaci-Dusseau
Memory Alignment: “C Interfaces and Implementations” Chapter 5 - David Hanson

Difficulty: Advanced Time estimate: 2 weeks Prerequisites: Strong C programming, understanding of pointers

Real world outcome:

// Replace libc malloc with yours
$ LD_PRELOAD=./mymalloc.so ./some_program

// Or link directly
$ gcc -o test test.c mymalloc.c

$ ./test
Allocated 1024 bytes at 0x7f1234000010
Allocated 2048 bytes at 0x7f1234000420
Freed 0x7f1234000010
Allocated 512 bytes at 0x7f1234000010 (reused!)
Coalesced free blocks: 1536 bytes available

Heap statistics:
  Total heap size: 65536 bytes
  Allocated: 2560 bytes
  Free: 62976 bytes (in 3 blocks)
  Fragmentation: 12%

Implementation Hints: Basic block structure:

struct block_header {
    size_t size;        // Size of this block (including header)
    int is_free;        // 1 if free, 0 if allocated
    struct block_header *next;  // Next block in free list
};

Allocation strategies:

First fit: Find first block that’s big enough (fast, more fragmentation)
Best fit: Find smallest block that fits (slower, less fragmentation)
Worst fit: Use largest block (sometimes good for preventing tiny fragments)

Coalescing: When freeing, check if adjacent blocks are also free. If so, merge them.

Learning milestones:

Basic alloc/free works → You understand heap structure
Coalescing prevents fragmentation → You understand memory management complexity
Your allocator passes real-world tests → You understand production requirements

Project 4: Virtual Memory Simulator

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: C
Alternative Programming Languages: Rust, Python (for simulation)
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
Difficulty: Level 4: Expert (The Systems Architect)
Knowledge Area: Virtual Memory / Paging
Software or Tool: VM Simulator
Main Book: “Operating Systems: Three Easy Pieces” by Arpaci-Dusseau

What you’ll build: A simulator that implements virtual memory with page tables, address translation, page faults, and page replacement algorithms (FIFO, LRU, Clock). Visualize what happens when programs access memory.

Why it teaches OS fundamentals: Virtual memory is arguably the most important OS abstraction. It gives each process the illusion of its own private address space, enables memory protection, allows more memory than physically exists (via swapping), and enables memory-mapped files. Understanding page tables and TLBs is essential.

Core challenges you’ll face:

Address translation (virtual → physical) → maps to page tables
Multi-level page tables → maps to memory efficiency
Page fault handling → maps to demand paging
Page replacement algorithms → maps to when physical memory is full
TLB simulation → maps to caching translations

Key Concepts:

Paging: “Operating Systems: Three Easy Pieces” Chapters 18-22 - Arpaci-Dusseau
Page Tables: “Computer Systems: A Programmer’s Perspective” Chapter 9 - Bryant & O’Hallaron
Page Replacement: “Operating Systems: Three Easy Pieces” Chapter 22 - Arpaci-Dusseau
TLB: “Computer Architecture” Chapter 5 - Hennessy & Patterson

Difficulty: Expert Time estimate: 2-3 weeks Prerequisites: Understanding of binary, addressing, basic OS concepts

Real world outcome:

$ ./vm_simulator workload.trace --pages=16 --frames=4 --algorithm=lru

Virtual Memory Simulator
========================
Virtual pages: 16 (64KB address space with 4KB pages)
Physical frames: 4 (16KB physical memory)
Algorithm: LRU (Least Recently Used)

Processing memory trace...

Access: 0x1234 → Page 1
  [Page fault! Loading page 1 into frame 0]
  Page table: [1→0, -, -, -, ...]
  Physical frames: [P1, -, -, -]

Access: 0x5678 → Page 5
  [Page fault! Loading page 5 into frame 1]
  Page table: [1→0, -, -, -, -, 5→1, ...]
  Physical frames: [P1, P5, -, -]

Access: 0x1240 → Page 1
  [TLB hit! Translating to frame 0]
  Physical address: 0x0240

...

Final Statistics:
  Total accesses:     10,000
  Page faults:        847 (8.47%)
  TLB hits:           8,234 (82.34%)

Comparison with other algorithms:
  FIFO:  1,024 faults (10.24%)
  LRU:   847 faults (8.47%)
  Clock: 891 faults (8.91%)
  OPT:   723 faults (7.23%) [theoretical minimum]

Implementation Hints: Page table entry structure:

struct pte {
    unsigned int present : 1;   // Is page in physical memory?
    unsigned int frame : 20;    // Physical frame number
    unsigned int dirty : 1;     // Has page been written?
    unsigned int accessed : 1;  // For LRU approximation
};

Address translation:

uint32_t translate(uint32_t virtual_addr) {
    uint32_t page_num = virtual_addr >> 12;      // Upper bits
    uint32_t offset = virtual_addr & 0xFFF;      // Lower 12 bits

    if (!page_table[page_num].present) {
        handle_page_fault(page_num);
    }

    uint32_t frame = page_table[page_num].frame;
    return (frame << 12) | offset;
}

For LRU: maintain access timestamps or use a list where accessed pages move to front.

Learning milestones:

Address translation works → You understand virtual → physical mapping
Page faults handled correctly → You understand demand paging
LRU beats FIFO → You understand replacement algorithm trade-offs

Project 5: Process Scheduler Simulator

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: C
Alternative Programming Languages: Rust, Python
Coolness Level: Level 3: Genuinely Clever
Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
Difficulty: Level 3: Advanced (The Engineer)
Knowledge Area: Process Management / Scheduling
Software or Tool: Scheduler Simulator
Main Book: “Operating Systems: Three Easy Pieces” by Arpaci-Dusseau

What you’ll build: A scheduler simulator that implements multiple scheduling algorithms (FIFO, SJF, Round Robin, MLFQ, CFS) and compares their performance on different workloads. Visualize process execution timelines.

Why it teaches OS fundamentals: The scheduler is the heart of multiprogramming—it decides which process runs when. Different algorithms optimize for different goals (throughput, latency, fairness). Understanding scheduling helps you understand why your program sometimes runs slowly even on a fast CPU.

Core challenges you’ll face:

Process state machine → maps to ready, running, blocked states
Context switch simulation → maps to overhead of switching
Priority implementation → maps to fairness vs efficiency
Multi-level queues → maps to different process types
Metrics calculation → maps to turnaround time, response time, fairness

Key Concepts:

Scheduling Algorithms: “Operating Systems: Three Easy Pieces” Chapters 7-9 - Arpaci-Dusseau
Process States: “Operating Systems: Three Easy Pieces” Chapter 4 - Arpaci-Dusseau
Multi-Level Feedback Queue: “Operating Systems: Three Easy Pieces” Chapter 8 - Arpaci-Dusseau
Completely Fair Scheduler: “Linux Kernel Development” Chapter 4 - Robert Love

Difficulty: Advanced Time estimate: 2 weeks Prerequisites: Understanding of processes, basic data structures

Real world outcome:

$ ./scheduler sim workload.txt --algorithm=mlfq

Process Scheduler Simulator
===========================
Workload: 5 processes
  P1: CPU burst 100ms
  P2: CPU burst 20ms, I/O 50ms, CPU burst 20ms
  P3: CPU burst 200ms
  P4: CPU burst 10ms
  P5: CPU burst 50ms

Algorithm: Multi-Level Feedback Queue (3 levels, quantum 10/20/40ms)

Timeline:
0ms    [P1 starts on Q0]
10ms   [P1 demoted to Q1] [P2 starts on Q0]
20ms   [P2 blocks on I/O]
20ms   [P3 starts on Q0]
30ms   [P3 demoted to Q1] [P4 starts on Q0]
40ms   [P4 completes!] [P5 starts on Q0]
...

Results:
          Turnaround  Response  Wait
  P1:     180ms       0ms       80ms
  P2:     110ms       10ms      40ms
  P3:     290ms       20ms      90ms
  P4:     40ms        30ms      30ms
  P5:     120ms       40ms      70ms

Average:  148ms       20ms      62ms

Comparison:
  Algorithm     Turnaround   Response   Fairness
  FIFO          220ms        72ms       Poor
  SJF           148ms        38ms       Poor (starvation)
  Round Robin   175ms        20ms       Good
  MLFQ          148ms        20ms       Good
  CFS           152ms        22ms       Excellent

Implementation Hints: Process Control Block (PCB):

struct process {
    int pid;
    enum { READY, RUNNING, BLOCKED, TERMINATED } state;
    int priority;
    int burst_remaining;
    int arrival_time;
    int start_time;  // For response time
    int completion_time;  // For turnaround time
};

MLFQ rules:

New processes start at highest priority queue
If process uses full quantum, demote to lower priority
If process blocks before quantum, stay at same priority
Periodic priority boost to prevent starvation

CFS (Linux’s scheduler) uses a red-black tree keyed by “virtual runtime”—processes that have run less get priority.

Learning milestones:

FIFO and SJF work → You understand basic scheduling
Round Robin achieves fairness → You understand time-slicing
MLFQ adapts to workload → You understand why modern schedulers are complex

Project 6: Fork and Exec (Process Creation)

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: C
Alternative Programming Languages: Rust (with libc bindings)
Coolness Level: Level 3: Genuinely Clever
Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
Difficulty: Level 2: Intermediate (The Developer)
Knowledge Area: Process Management / Unix API
Software or Tool: Process Explorer
Main Book: “The Linux Programming Interface” by Michael Kerrisk

What you’ll build: A tool that explores and demonstrates Unix process creation: forking, exec’ing, waiting, orphans, zombies, and the process tree. Understand the elegance of fork+exec separation.

Why it teaches OS fundamentals: Unix’s fork() + exec() model is fundamentally different from Windows’ CreateProcess(). Understanding this separation (fork = copy, exec = replace) reveals Unix’s philosophy: simple primitives that compose. This is why Unix shells are so powerful.

Core challenges you’ll face:

Understanding fork’s return value → maps to parent vs child distinction
Exec family (execve, execlp, etc.) → maps to program replacement
Wait and exit status → maps to parent-child synchronization
Orphan and zombie processes → maps to process lifecycle edge cases
Building a process tree visualizer → maps to process hierarchy

Key Concepts:

fork(): “The Linux Programming Interface” Chapter 24 - Michael Kerrisk
exec(): “The Linux Programming Interface” Chapter 27 - Michael Kerrisk
Process Relationships: “Advanced Programming in the UNIX Environment” Chapter 8 - Stevens & Rago
Unix vs Windows Process Creation: “Operating Systems: Three Easy Pieces” Chapter 5 - Arpaci-Dusseau

Difficulty: Intermediate Time estimate: 1 week Prerequisites: Basic C programming, familiarity with Unix

Real world outcome:

// fork_explorer output
$ ./fork_explorer

=== Demonstrating fork() ===
Parent PID: 1234
Calling fork()...

[Parent] fork() returned 1235 (child's PID)
[Child]  fork() returned 0 (I am the child)

=== Demonstrating fork + exec ===
Forking, then exec'ing /bin/ls...

[Parent] Waiting for child 1236...
[Child]  About to exec /bin/ls
file1.txt  file2.txt  fork_explorer
[Parent] Child 1236 exited with status 0

=== Process tree ===
$ ./fork_explorer tree

init(1)
├── systemd(456)
│   ├── sshd(789)
│   │   └── bash(1234)
│   │       └── fork_explorer(1235)
│   └── cron(790)
└── ...

=== Demonstrating zombies ===
Forking child that exits immediately...
Child 1237 has exited
[Before wait] ps shows: 1237 Z+ (zombie!)
[After wait] Zombie reaped

=== Why fork+exec is elegant ===
Want to redirect output? fork, then redirect, then exec.
Want to change directory? fork, then chdir, then exec.
The child can modify its own environment before exec!

Implementation Hints: The fork+exec pattern:

pid_t pid = fork();
if (pid == 0) {
    // Child: set up environment, then become another program
    close(1);  // Close stdout
    open("output.txt", O_WRONLY);  // Now fd 1 is the file!
    execl("/bin/ls", "ls", "-l", NULL);
    // exec doesn't return on success
    perror("exec failed");
    exit(1);
} else {
    // Parent: wait for child
    int status;
    waitpid(pid, &status, 0);
}

This is how shell redirection (ls > file) works! Redirect after fork, before exec.

Learning milestones:

You understand fork’s magic → Parent and child continue from the same point
You understand exec’s transformation → Process image completely replaced
You see the elegance → Separation enables shell features like redirection

Project 7: Unix Shell from Scratch

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: C
Alternative Programming Languages: Rust, Go
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 2. The “Micro-SaaS / Pro Tool” (Solo-Preneur Potential)
Difficulty: Level 3: Advanced (The Engineer)
Knowledge Area: Process Management / Unix Philosophy
Software or Tool: Unix Shell
Main Book: “The Linux Programming Interface” by Michael Kerrisk

What you’ll build: A fully functional Unix shell with command execution, pipes, I/O redirection, background jobs, signal handling, and built-in commands. This is the ultimate Unix process management project.

Why it teaches OS fundamentals: The shell is where Unix philosophy comes alive. Every feature you implement teaches a fundamental concept: fork+exec for commands, pipes for IPC, redirection for file I/O, signals for job control. Building a shell teaches you how Unix really works.

Core challenges you’ll face:

Parsing command lines → maps to lexing and parsing
**Implementing pipes ( )** → maps to inter-process communication
I/O redirection (<, >, ») → maps to file descriptors
Background jobs (&) → maps to job control and signals
Signal handling (Ctrl+C, Ctrl+Z) → maps to asynchronous events

Key Concepts:

Pipes: “The Linux Programming Interface” Chapter 44 - Michael Kerrisk
I/O Redirection: “Advanced Programming in the UNIX Environment” Chapter 3 - Stevens & Rago
Signals: “The Linux Programming Interface” Chapters 20-22 - Michael Kerrisk
Job Control: “Advanced Programming in the UNIX Environment” Chapter 9 - Stevens & Rago

Difficulty: Advanced Time estimate: 3-4 weeks Prerequisites: Project 6, strong C skills

Real world outcome:

$ ./myshell
myshell> echo Hello, World!
Hello, World!

myshell> ls -la | grep .c | wc -l
5

myshell> cat input.txt > output.txt

myshell> sort < unsorted.txt | uniq > sorted.txt

myshell> sleep 100 &
[1] 1234

myshell> jobs
[1]  Running    sleep 100

myshell> fg 1
sleep 100
^C
[1]  Interrupted    sleep 100

myshell> cd /tmp
myshell> pwd
/tmp

myshell> export PATH=$PATH:/custom/bin

myshell> history
    1  echo Hello, World!
    2  ls -la | grep .c | wc -l
    ...

myshell> exit
Goodbye!

Implementation Hints: Pipeline implementation:

// For: cmd1 | cmd2
int pipefd[2];
pipe(pipefd);  // pipefd[0]=read, pipefd[1]=write

if (fork() == 0) {
    // cmd1: write to pipe
    close(pipefd[0]);
    dup2(pipefd[1], STDOUT_FILENO);
    close(pipefd[1]);
    exec(cmd1);
}

if (fork() == 0) {
    // cmd2: read from pipe
    close(pipefd[1]);
    dup2(pipefd[0], STDIN_FILENO);
    close(pipefd[0]);
    exec(cmd2);
}

// Parent closes both ends and waits
close(pipefd[0]);
close(pipefd[1]);
wait(NULL); wait(NULL);

For longer pipelines, chain multiple pipes. For job control, use setpgid() and tcsetpgrp().

Learning milestones:

Basic commands execute → You understand fork+exec deeply
Pipes work → You understand IPC
Job control works → You understand signals and process groups
You appreciate bash → You understand how complex a real shell is

Project 8: System Call Interface

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: C + Assembly
Alternative Programming Languages: Rust (with inline asm)
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
Difficulty: Level 4: Expert (The Systems Architect)
Knowledge Area: Kernel Interface / System Programming
Software or Tool: Syscall Explorer
Main Book: “The Linux Programming Interface” by Michael Kerrisk

What you’ll build: A tool that makes system calls directly (bypassing libc), traces system calls made by programs, and implements a simple syscall wrapper library. Understand the user-kernel boundary.

Why it teaches OS fundamentals: System calls are the OS’s API—the only way user programs can request kernel services. Understanding how the CPU transitions from user mode to kernel mode, how arguments are passed, and how results are returned reveals the true interface between you and the OS.

Core challenges you’ll face:

Making raw syscalls without libc → maps to syscall ABI
Understanding the user/kernel boundary → maps to privilege levels
Tracing syscalls (like strace) → maps to ptrace and debugging
Implementing a syscall wrapper → maps to what libc actually does
Comparing Linux and Windows syscalls → maps to OS API design

Key Concepts:

System Calls: “The Linux Programming Interface” Chapter 3 - Michael Kerrisk
x86-64 Syscall Convention: “Low-Level Programming” Chapter 8 - Igor Zhirkov
ptrace for Tracing: “The Linux Programming Interface” Chapter 26 - Michael Kerrisk
User/Kernel Transition: “Linux Kernel Development” Chapter 5 - Robert Love

Difficulty: Expert Time estimate: 2 weeks Prerequisites: Assembly knowledge, understanding of kernel concepts

Real world outcome:

// syscall_explorer output

$ ./syscall_explorer raw

=== Making raw syscalls (no libc) ===

// write(1, "Hello\n", 6) using raw syscall
movq $1, %rax      // syscall number (write = 1)
movq $1, %rdi      // fd = 1 (stdout)
movq $msg, %rsi    // buffer
movq $6, %rdx      // count
syscall            // CPU transitions to kernel mode!

Result: Hello
Returned: 6 (bytes written)

$ ./syscall_explorer trace /bin/ls

=== Tracing syscalls of /bin/ls ===
execve("/bin/ls", ["ls"], [/* 50 vars */]) = 0
brk(NULL)                               = 0x5555557a0000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT
openat(AT_FDCWD, "/lib/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF...", 832)              = 832
mmap(NULL, 1847776, PROT_READ)          = 0x7f1234000000
...
write(1, "file1.txt\nfile2.txt\n", 20)  = 20
close(1)                                = 0
exit_group(0)                           = ?

Total: 42 syscalls

=== Linux vs Windows ===
Operation          Linux Syscall    Windows API
---------          -------------    -----------
Write to file      write()          NtWriteFile()
Create process     fork()+exec()    NtCreateProcess()
Open file          openat()         NtCreateFile()
Memory allocation  mmap()           NtAllocateVirtualMemory()

Linux: ~450 syscalls, simple interface
Windows: ~400 NT syscalls, more complex, often wrapped by Win32 API

Implementation Hints: x86-64 Linux syscall convention:

rax: syscall number
rdi, rsi, rdx, r10, r8, r9: arguments 1-6
syscall instruction transitions to kernel
rax: return value

Raw syscall in inline assembly:

long my_write(int fd, const void *buf, size_t count) {
    long ret;
    asm volatile (
        "syscall"
        : "=a" (ret)
        : "a" (1), "D" (fd), "S" (buf), "d" (count)  // 1 = write
        : "rcx", "r11", "memory"
    );
    return ret;
}

For tracing, use ptrace(PTRACE_SYSCALL, ...) to stop at each syscall entry/exit.

Learning milestones:

Raw syscall works → You understand the syscall ABI
Tracer shows all syscalls → You understand what programs really do
You see the OS’s true interface → You understand the kernel boundary

Project 9: File System from Scratch

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: C
Alternative Programming Languages: Rust, C++
Coolness Level: Level 5: Pure Magic (Super Cool)
Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
Difficulty: Level 4: Expert (The Systems Architect)
Knowledge Area: File Systems / Storage
Software or Tool: Custom File System
Main Book: “Operating Systems: Three Easy Pieces” by Arpaci-Dusseau

What you’ll build: A simple file system (like ext2 or FAT) that runs in a disk image file. Implement superblock, inodes, data blocks, directories, and basic operations (create, read, write, delete, list). Then mount it with FUSE!

Why it teaches OS fundamentals: File systems are how the OS organizes persistent storage. Understanding inodes, block allocation, directories as special files, and journaling for crash recovery reveals how “files” become bytes on disk and back again.

Core challenges you’ll face:

Block layout design → maps to disk organization
Inode structure → maps to file metadata
Directory implementation → maps to name → inode mapping
Block allocation (bitmap or free list) → maps to space management
Path traversal → maps to hierarchical namespace

Key Concepts:

File System Implementation: “Operating Systems: Three Easy Pieces” Chapters 39-42 - Arpaci-Dusseau
Inodes and Directories: “Understanding the Linux Kernel” Chapter 12 - Bovet & Cesati
FUSE Interface: “The Linux Programming Interface” Chapter 14 - Michael Kerrisk
Crash Consistency: “Operating Systems: Three Easy Pieces” Chapter 42 - Arpaci-Dusseau

Difficulty: Expert Time estimate: 4-6 weeks Prerequisites: Strong C, understanding of file concepts

Real world outcome:

$ ./mkfs.myfs disk.img 10M

Creating MyFS filesystem on disk.img (10 MB)
Block size: 4096 bytes
Total blocks: 2560
Inode count: 256
Data blocks: 2048
Reserved: 512 (superblock, bitmaps, inode table)

Filesystem created!

$ ./fsck.myfs disk.img
MyFS filesystem check
Superblock: OK
Inode bitmap: OK
Block bitmap: OK
Root directory: OK
Filesystem clean.

$ ./myfs disk.img /mnt/myfs    # Mount with FUSE!

$ cd /mnt/myfs
$ echo "Hello, MyFS!" > hello.txt
$ cat hello.txt
Hello, MyFS!

$ mkdir subdir
$ ls -la
total 8
drwxr-xr-x  3 user user 4096 Jan 1 12:00 .
drwxr-xr-x  5 user user 4096 Jan 1 12:00 ..
-rw-r--r--  1 user user   14 Jan 1 12:00 hello.txt
drwxr-xr-x  2 user user 4096 Jan 1 12:00 subdir

$ # Raw disk dump
$ ./myfs-debug disk.img

Superblock:
  Magic: 0xMYFS
  Block size: 4096
  Inode count: 256
  Free inodes: 253
  Free blocks: 2044

Inode 2 (root directory):
  Type: directory
  Size: 4096
  Blocks: [100]
  Permissions: 0755

Block 100 (root directory data):
  Entry 0: "."      → inode 2
  Entry 1: ".."     → inode 2
  Entry 2: "hello.txt" → inode 3
  Entry 3: "subdir" → inode 4

Implementation Hints: Disk layout:

Block 0: Superblock (magic, sizes, free counts)
Block 1: Inode bitmap (1 bit per inode)
Block 2: Block bitmap (1 bit per data block)
Block 3-N: Inode table (inodes, ~128 bytes each)
Block N+: Data blocks

Inode structure:

struct inode {
    uint16_t mode;        // Permissions + type
    uint16_t uid, gid;    // Owner
    uint32_t size;        // File size
    uint32_t atime, mtime, ctime;
    uint32_t blocks[12];  // Direct block pointers
    uint32_t indirect;    // Indirect block pointer
};

Directories are just files containing (name, inode) pairs.

Learning milestones:

Create and read files → You understand blocks and inodes
Directories work → You understand the namespace abstraction
FUSE mount works → You’ve built a real, usable file system!

Project 10: Everything Is a File (Device Files)

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: C
Alternative Programming Languages: Rust
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
Difficulty: Level 3: Advanced (The Engineer)
Knowledge Area: Device Drivers / Unix Philosophy
Software or Tool: Virtual Devices
Main Book: “Linux Device Drivers” by Corbet, Rubini & Kroah-Hartman

What you’ll build: A set of virtual device drivers using FUSE or kernel modules that implement the Unix “everything is a file” philosophy: a random number device, a counter device, a virtual sensor, and more.

Why it teaches OS fundamentals: The “everything is a file” abstraction is Unix’s killer feature. By implementing your own character devices, you’ll understand why this abstraction is so powerful: any program that can read/write files can interact with any device, no special APIs needed.

Core challenges you’ll face:

Character vs block devices → maps to device types
Implementing read/write handlers → maps to device operations
Device state management → maps to driver complexity
Creating device nodes → maps to /dev filesystem
Understanding major/minor numbers → maps to device identification

Key Concepts:

Device Drivers: “Linux Device Drivers” Chapters 1-3 - Corbet et al.
Character Devices: “Linux Device Drivers” Chapter 3 - Corbet et al.
/dev and udev: “The Linux Programming Interface” Chapter 14 - Michael Kerrisk
FUSE for Userspace Devices: FUSE documentation

Difficulty: Advanced Time estimate: 2 weeks Prerequisites: C programming, basic kernel concepts

Real world outcome:

$ # Your custom devices in /dev!

$ cat /dev/myrandom  # Your random number generator
8b2e7f4a1c3d9e6b

$ echo 100 > /dev/mycounter  # Set counter
$ cat /dev/mycounter
100
$ cat /dev/mycounter
101
$ cat /dev/mycounter
102

$ cat /dev/mytemp    # Virtual temperature sensor
23.5

$ echo "Hello" > /dev/myupper
$ cat /dev/myupper
HELLO

$ # Composability in action!
$ cat /dev/myrandom | xxd | head -5
00000000: 3862 3265 3766 3461 3163 3364 3965 3662  8b2e7f4a1c3d9e6b

$ # Works with ANY program that reads files!
$ python3 -c "print(open('/dev/mycounter').read())"
105

Implementation Hints: Using FUSE (simpler, userspace):

static int my_read(const char *path, char *buf, size_t size,
                   off_t offset, struct fuse_file_info *fi) {
    if (strcmp(path, "/random") == 0) {
        // Generate random hex string
        int len = snprintf(buf, size, "%08x\n", rand());
        return len;
    }
    if (strcmp(path, "/counter") == 0) {
        static int counter = 0;
        int len = snprintf(buf, size, "%d\n", counter++);
        return len;
    }
    return -ENOENT;
}

static struct fuse_operations my_ops = {
    .read = my_read,
    .write = my_write,
    .getattr = my_getattr,
};

For a kernel module (harder, more powerful):

static ssize_t dev_read(struct file *f, char __user *buf,
                        size_t len, loff_t *off) {
    // Copy data to userspace
    copy_to_user(buf, data, len);
    return len;
}

static struct file_operations fops = {
    .read = dev_read,
    .write = dev_write,
};

Learning milestones:

Devices appear in /dev → You understand device files
Read/write work → You understand the file abstraction for devices
Standard tools work with your devices → You understand composability

Project 11: Inter-Process Communication

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: C
Alternative Programming Languages: Rust, Go
Coolness Level: Level 3: Genuinely Clever
Business Potential: 2. The “Micro-SaaS / Pro Tool” (Solo-Preneur Potential)
Difficulty: Level 3: Advanced (The Engineer)
Knowledge Area: IPC / Concurrency
Software or Tool: IPC Explorer
Main Book: “The Linux Programming Interface” by Michael Kerrisk

What you’ll build: A comprehensive IPC explorer that demonstrates all Unix IPC mechanisms: pipes, named pipes (FIFOs), message queues, shared memory, semaphores, and Unix domain sockets. Compare their performance and use cases.

Why it teaches OS fundamentals: Processes are isolated by design (that’s protection!), but they often need to communicate. Each IPC mechanism has different trade-offs: pipes are simple, shared memory is fast, sockets are flexible. Understanding IPC is essential for building multi-process systems.

Core challenges you’ll face:

Pipes and FIFOs → maps to byte streams between processes
Shared memory → maps to zero-copy communication
Semaphores → maps to synchronization
Message queues → maps to typed message passing
Unix domain sockets → maps to socket API for local IPC

Key Concepts:

Pipes and FIFOs: “The Linux Programming Interface” Chapters 44-45 - Michael Kerrisk
Shared Memory: “The Linux Programming Interface” Chapters 48-50 - Michael Kerrisk
Semaphores: “The Linux Programming Interface” Chapters 47, 53 - Michael Kerrisk
Unix Domain Sockets: “The Linux Programming Interface” Chapter 57 - Michael Kerrisk

Difficulty: Advanced Time estimate: 2-3 weeks Prerequisites: Project 7, understanding of process concepts

Real world outcome:

$ ./ipc_explorer

=== Inter-Process Communication Comparison ===

Testing with 1 million 64-byte messages:

1. Pipe (unidirectional)
   Throughput: 2.3 GB/s
   Latency:    1.2 µs
   Use case:   Simple parent-child communication

2. Named Pipe (FIFO)
   Throughput: 2.1 GB/s
   Latency:    1.5 µs
   Use case:   Unrelated processes, persistent

3. Message Queue (POSIX)
   Throughput: 0.8 GB/s
   Latency:    3.2 µs
   Use case:   Priority messages, typed data

4. Shared Memory + Semaphore
   Throughput: 12.4 GB/s (!)
   Latency:    0.3 µs
   Use case:   High-performance, complex synchronization

5. Unix Domain Socket
   Throughput: 1.9 GB/s
   Latency:    1.8 µs
   Use case:   Bidirectional, socket API, file descriptor passing

=== Demo: Producer-Consumer with each mechanism ===

$ ./ipc_demo pipe
[Producer] Sending: "Hello from producer"
[Consumer] Received: "Hello from producer"

$ ./ipc_demo shm
[Producer] Wrote to shared memory at 0x7f1234000000
[Consumer] Read from shared memory: "Hello from producer"

$ ./ipc_demo socket
[Server] Listening on /tmp/ipc.sock
[Client] Connected
[Server] Received: "Hello from client"
[Client] Received response: "Hello from server"

Implementation Hints: Shared memory with semaphore:

// Create shared memory
int fd = shm_open("/myshm", O_CREAT | O_RDWR, 0666);
ftruncate(fd, SIZE);
void *ptr = mmap(NULL, SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);

// Create semaphore for synchronization
sem_t *sem = sem_open("/mysem", O_CREAT, 0666, 0);

// Producer writes, then signals
memcpy(ptr, data, len);
sem_post(sem);

// Consumer waits, then reads
sem_wait(sem);
memcpy(local, ptr, len);

Unix domain sockets work like TCP but on the same machine—much faster and can pass file descriptors between processes!

Learning milestones:

Pipes work for basic IPC → You understand byte stream communication
Shared memory is much faster → You understand zero-copy trade-offs
You can choose the right IPC → You understand when to use each mechanism

Project 12: Unix Philosophy Tools

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: C
Alternative Programming Languages: Rust, Go
Coolness Level: Level 3: Genuinely Clever
Business Potential: 2. The “Micro-SaaS / Pro Tool” (Solo-Preneur Potential)
Difficulty: Level 2: Intermediate (The Developer)
Knowledge Area: Unix Philosophy / Text Processing
Software or Tool: Core Utilities
Main Book: “The Linux Command Line” by William Shotts

What you’ll build: Implement your own versions of classic Unix tools: cat, head, tail, wc, grep, sort, uniq, cut, tr. Then use them together in pipelines to see Unix philosophy in action.

Why it teaches OS fundamentals: These tools embody Unix philosophy: each does one thing well, reads from stdin, writes to stdout, and can be composed. Building them teaches you about file I/O, text processing, and why this simple pattern is so powerful.

Core challenges you’ll face:

Standard I/O handling → maps to stdin/stdout/stderr
Command-line argument parsing → maps to program interface
Efficient text processing → maps to buffered I/O
Filter pattern implementation → maps to Unix composability
Making them work in pipelines → maps to process cooperation

Key Concepts:

Standard I/O: “The Linux Programming Interface” Chapter 4 - Michael Kerrisk
Unix Philosophy: “The Art of Unix Programming” Chapter 1 - Eric S. Raymond
Text Processing: “The Linux Command Line” Chapters 6, 19 - William Shotts
Filter Programs: “Advanced Programming in the UNIX Environment” Chapter 15 - Stevens & Rago

Difficulty: Intermediate Time estimate: 2-3 weeks Prerequisites: C programming, understanding of I/O

Real world outcome:

$ # Your implementations work just like the real ones!

$ ./mycat file1.txt file2.txt
[Contents of file1.txt]
[Contents of file2.txt]

$ ./myhead -n 5 bigfile.txt
[First 5 lines]

$ ./mywc -l *.c
  125 main.c
   87 utils.c
  212 total

$ ./mygrep "error" log.txt
Line 45: error: file not found
Line 123: error: permission denied

$ # The magic: composability!
$ ./mycat access.log | ./mygrep "404" | ./mycut -d' ' -f7 | ./mysort | ./myuniq -c | ./mysort -rn | ./myhead -5

  1523 /missing-page.html
   892 /old-api/users
   445 /favicon.ico
   234 /wp-admin.php
   122 /api/v1/deprecated

$ # That's 6 of YOUR programs working together!
$ # Each did one thing. The shell connected them.
$ # This is Unix philosophy in action.

Implementation Hints: Filter program template:

int main(int argc, char *argv[]) {
    FILE *input = stdin;

    // If filename given, use that
    if (argc > 1) {
        input = fopen(argv[1], "r");
        if (!input) { perror(argv[1]); return 1; }
    }

    char line[4096];
    while (fgets(line, sizeof(line), input)) {
        // Process line
        // Write to stdout
        fputs(processed, stdout);
    }

    if (input != stdin) fclose(input);
    return 0;
}

This pattern works for cat, grep, sort, uniq, tr, etc. The key insight: stdin/stdout are the universal interface.

Learning milestones:

Individual tools work → You understand file I/O
They compose in pipelines → You understand Unix philosophy
You see the elegance → Small tools + shell = infinite power

Project 13: Containers from Scratch

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: C
Alternative Programming Languages: Go, Rust
Coolness Level: Level 5: Pure Magic (Super Cool)
Business Potential: 4. The “Open Core” Infrastructure (Enterprise Scale)
Difficulty: Level 4: Expert (The Systems Architect)
Knowledge Area: Namespaces / Isolation
Software or Tool: Container Runtime
Main Book: “Container Security” by Liz Rice

What you’ll build: A minimal container runtime that uses Linux namespaces and cgroups to isolate processes—essentially, a mini-Docker. Understand how containers are NOT virtual machines, but isolated Linux processes.

Why it teaches OS fundamentals: Containers are the most important infrastructure innovation of the last decade, and they’re pure Unix/Linux magic. They use kernel features (namespaces for isolation, cgroups for resource limits) to create the illusion of separate systems. Understanding containers means understanding modern OS isolation.

Core challenges you’ll face:

PID namespace → maps to isolated process tree
Mount namespace → maps to isolated filesystem
Network namespace → maps to isolated network stack
User namespace → maps to uid/gid mapping
cgroups → maps to resource limits (CPU, memory)

Key Concepts:

Namespaces: “Container Security” Chapter 4 - Liz Rice
cgroups: “Container Security” Chapter 5 - Liz Rice
chroot and pivot_root: “The Linux Programming Interface” Chapter 18 - Michael Kerrisk
Linux Capabilities: “Container Security” Chapter 7 - Liz Rice

Difficulty: Expert Time estimate: 3-4 weeks Prerequisites: Deep Linux knowledge, C programming

Real world outcome:

$ sudo ./mycontainer run alpine /bin/sh

=== Creating container ===
[+] Creating namespaces: pid, mnt, uts, net, user
[+] Setting up cgroups: 512MB memory limit, 50% CPU
[+] Setting hostname to "container"
[+] Mounting container filesystem (alpine rootfs)
[+] Pivoting root
[+] Dropping capabilities
[+] Executing /bin/sh

/ # hostname
container

/ # ps aux
PID   USER     TIME  COMMAND
    1 root      0:00 /bin/sh   ← We are PID 1!
    2 root      0:00 ps aux

/ # cat /etc/os-release
NAME="Alpine Linux"    ← Different from host!

/ # ip addr
1: lo: <LOOPBACK> ...
   inet 127.0.0.1/8    ← Isolated network

/ # # Try to escape... nope!
/ # ls /host-root
ls: /host-root: No such file or directory

$ # From the host, it's just a process:
$ ps aux | grep mycontainer
root  12345  0.0  0.1  ... ./mycontainer run alpine /bin/sh

Implementation Hints: Container creation with namespaces:

int child_func(void *arg) {
    // We're in the new namespaces now!

    // Set hostname
    sethostname("container", 9);

    // Mount new root
    mount("none", "/", NULL, MS_REC | MS_PRIVATE, NULL);
    mount(rootfs, rootfs, "bind", MS_BIND | MS_REC, NULL);

    // Pivot root
    mkdir(put_old, 0700);
    pivot_root(rootfs, put_old);

    // Unmount old root
    umount2(put_old, MNT_DETACH);

    // Execute command
    execv(cmd, args);
}

int main() {
    int flags = CLONE_NEWPID | CLONE_NEWNS | CLONE_NEWUTS |
                CLONE_NEWNET | CLONE_NEWUSER | SIGCHLD;

    char *stack = malloc(STACK_SIZE) + STACK_SIZE;
    pid_t pid = clone(child_func, stack, flags, arg);

    // Set up cgroups for resource limits
    // ...

    waitpid(pid, NULL, 0);
}

cgroups: write to /sys/fs/cgroup/*/container-name/ to limit resources.

Learning milestones:

Process isolation works → You understand namespaces
Resource limits work → You understand cgroups
You understand Docker → Containers are just fancy processes!

Project 14: xv6 Operating System Study

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: C
Alternative Programming Languages: N/A (xv6 is specifically C)
Coolness Level: Level 5: Pure Magic (Super Cool)
Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
Difficulty: Level 4: Expert (The Systems Architect)
Knowledge Area: Complete OS / MIT Teaching OS
Software or Tool: xv6 OS
Main Book: “xv6: a simple, Unix-like teaching operating system” (MIT)

What you’ll build: Study and modify xv6, MIT’s teaching operating system. Add a new system call, implement a new scheduling algorithm, add a new shell feature. Understand a complete, working Unix-like OS.

Why it teaches OS fundamentals: xv6 is a complete, readable, runnable Unix-like OS in about 10,000 lines of C. It has everything: boot, processes, memory, file system, pipes, shell. Unlike reading Linux source, xv6 is designed to be understood. This is the “see it all working together” project.

Core challenges you’ll face:

Understanding the boot process → maps to how OS starts
Adding a system call → maps to kernel interface
Modifying the scheduler → maps to CPU allocation
Understanding the file system → maps to persistent storage
Debugging kernel code → maps to systems debugging

Key Concepts:

Complete OS Structure: “xv6 Book” (MIT) - All chapters
Unix Implementation: “The Design of the Unix Operating System” - Maurice Bach
Kernel Debugging: “Operating Systems: Three Easy Pieces” Appendix - Arpaci-Dusseau
OS Architecture: “Operating Systems: Three Easy Pieces” - Arpaci-Dusseau

Difficulty: Expert Time estimate: 4-8 weeks Prerequisites: Most previous projects, strong C skills

Real world outcome:

$ cd xv6-riscv
$ make qemu

xv6 kernel is booting
hart 1 starting
hart 2 starting
init: starting sh

$ # You're inside xv6!
$ ls
.              1 1 1024
..             1 1 1024
README         2 2 2286
cat            2 3 23896
echo           2 4 22720
...

$ # Your modifications:

$ # 1. New system call: getcount (process syscall count)
$ getcount
Process 3: 47 system calls

$ # 2. New scheduler: lottery scheduling
$ ps
PID  TICKETS  STATE    NAME
1    10       running  init
2    20       ready    sh
3    5        sleeping cat

$ # 3. New shell feature: command history
$ history
1: ls
2: cat README
3: getcount

$ # Trace a system call through the whole stack:
# User calls write()
# → trap into kernel (ecall instruction)
# → syscall dispatcher looks up sys_write
# → sys_write validates and calls filewrite
# → filewrite calls the inode layer
# → inode layer calls the block layer
# → block layer calls the disk driver
# → bytes reach the disk!

Implementation Hints: Adding a system call:

Add syscall number to kernel/syscall.h
Add handler to kernel/syscall.c
Implement the handler in a kernel file
Add user-space wrapper in user/user.h and user/usys.pl

File system structure:

Block 0:    Boot sector
Block 1:    Superblock
Block 2-31: Inode blocks
Block 32+:  Bitmap + data blocks

Key files to study:

kernel/main.c: Boot sequence
kernel/trap.c: Interrupt/syscall handling
kernel/proc.c: Process management
kernel/vm.c: Virtual memory
kernel/fs.c: File system
kernel/bio.c: Block I/O

Learning milestones:

You understand the boot process → Follow main.c through startup
You can add a system call → You understand the user/kernel boundary
You understand the file system → You can trace read() from user to disk
You can modify the scheduler → You understand process management

Project 15: Kernel Module Development

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: C
Alternative Programming Languages: Rust (for Linux kernel)
Coolness Level: Level 5: Pure Magic (Super Cool)
Business Potential: 3. The “Service & Support” Model (B2B Utility)
Difficulty: Level 4: Expert (The Systems Architect)
Knowledge Area: Kernel Development / Device Drivers
Software or Tool: Linux Kernel Module
Main Book: “Linux Device Drivers” by Corbet, Rubini & Kroah-Hartman

What you’ll build: A Linux kernel module that adds functionality to the running kernel: a character device, a proc file, a system call, or a network filter. Experience actual kernel programming.

Why it teaches OS fundamentals: Kernel modules let you extend Linux without recompiling. You’ll experience kernel programming: no libc, no user-space safety, one bug = kernel panic. This is OS development in the real world.

Core challenges you’ll face:

Kernel build system → maps to kernel development workflow
Kernel memory allocation → maps to kmalloc, no malloc
Kernel synchronization → maps to spinlocks, mutexes in kernel
Proc filesystem interface → maps to /proc as kernel-user bridge
Surviving kernel crashes → maps to debugging without printf

Key Concepts:

Module Basics: “Linux Device Drivers” Chapter 2 - Corbet et al.
Character Devices: “Linux Device Drivers” Chapter 3 - Corbet et al.
Proc Filesystem: “Linux Device Drivers” Chapter 4 - Corbet et al.
Kernel Debugging: “Linux Kernel Development” Chapter 18 - Robert Love

Difficulty: Expert Time estimate: 3-4 weeks Prerequisites: Strong C, understanding of kernel concepts

Real world outcome:

$ # Build and load your module
$ make
$ sudo insmod mymodule.ko

$ dmesg | tail -3
[12345.678] mymodule: Hello from the kernel!
[12345.679] mymodule: Created /dev/mydev and /proc/myinfo

$ # Your character device
$ echo "test" > /dev/mydev
$ cat /dev/mydev
Received: test (5 bytes)

$ # Your proc file
$ cat /proc/myinfo
MyModule Statistics
-------------------
Opens: 3
Reads: 15
Writes: 2
Bytes processed: 1234

$ # Your module is part of the kernel!
$ lsmod | grep mymodule
mymodule    16384  0

$ # Clean up
$ sudo rmmod mymodule
$ dmesg | tail -1
[12400.123] mymodule: Goodbye from the kernel!

Implementation Hints: Basic kernel module:

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>

static int __init mymodule_init(void) {
    printk(KERN_INFO "mymodule: Hello!\n");
    // Create devices, register handlers, etc.
    return 0;
}

static void __exit mymodule_exit(void) {
    printk(KERN_INFO "mymodule: Goodbye!\n");
    // Clean up
}

module_init(mymodule_init);
module_exit(mymodule_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("You");
MODULE_DESCRIPTION("My first kernel module");

Key differences from user space:

No libc: use kernel functions (kmalloc, not malloc)
No floating point!
Sleep carefully: msleep, not sleep
Concurrency is harder: use spinlocks, RCU, etc.

Learning milestones:

Module loads and unloads → You understand the kernel module system
Device or proc file works → You’ve written kernel code that users can interact with
You survive kernel crashes gracefully → You’re a kernel developer!

Project 16: Network Stack Exploration

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: C
Alternative Programming Languages: Rust, Go
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
Difficulty: Level 4: Expert (The Systems Architect)
Knowledge Area: Networking / Protocol Implementation
Software or Tool: Network Stack
Main Book: “TCP/IP Illustrated, Volume 1” by W. Richard Stevens

What you’ll build: Implement a minimal TCP/IP stack: raw socket handling, IP packet parsing, ARP, ICMP (ping), and basic TCP (connection establishment). Understand how network “just works.”

Why it teaches OS fundamentals: Networking is a core OS responsibility. Understanding how packets flow from application to wire and back—through sockets, protocol layers, and device drivers—reveals one of the OS’s most complex subsystems. The network stack is also where “everything is a file” meets real-world complexity.

Core challenges you’ll face:

Raw socket programming → maps to bypassing the kernel stack
Ethernet frame handling → maps to layer 2
IP packet parsing/creation → maps to layer 3
ICMP implementation → maps to network diagnostics
TCP state machine → maps to reliable transport

Key Concepts:

TCP/IP Layers: “TCP/IP Illustrated, Volume 1” Chapter 1 - Stevens
IP Protocol: “TCP/IP Illustrated, Volume 1” Chapters 5-8 - Stevens
TCP Protocol: “TCP/IP Illustrated, Volume 1” Chapters 17-24 - Stevens
Raw Sockets: “The Linux Programming Interface” Chapter 58 - Michael Kerrisk

Difficulty: Expert Time estimate: 4-6 weeks Prerequisites: Strong C, understanding of networking concepts

Real world outcome:

$ # Your minimal TCP/IP stack!

$ sudo ./mynet interface tap0

=== MyNet TCP/IP Stack ===
Listening on tap0 (10.0.0.2/24)

$ # From another terminal:
$ ping 10.0.0.2

[MyNet] Received ARP request: Who has 10.0.0.2?
[MyNet] Sending ARP reply: 10.0.0.2 is at aa:bb:cc:dd:ee:ff
[MyNet] Received ICMP Echo Request from 10.0.0.1
[MyNet] Sending ICMP Echo Reply to 10.0.0.1
[MyNet] Received ICMP Echo Request from 10.0.0.1
[MyNet] Sending ICMP Echo Reply to 10.0.0.1
^C

$ # TCP connection!
$ nc 10.0.0.2 8080 <<< "Hello"

[MyNet] Received TCP SYN from 10.0.0.1:45678 → :8080
[MyNet] Sending TCP SYN+ACK
[MyNet] Received TCP ACK - Connection established!
[MyNet] TCP state: ESTABLISHED
[MyNet] Received TCP data: "Hello\n" (6 bytes)
[MyNet] Sending TCP ACK
[MyNet] Received TCP FIN
[MyNet] Sending TCP FIN+ACK
[MyNet] TCP state: CLOSED

$ # See the packet structure:
$ ./mynet dump

IP Header:
  Version: 4
  IHL: 5 (20 bytes)
  Total Length: 60
  TTL: 64
  Protocol: 6 (TCP)
  Source: 10.0.0.1
  Destination: 10.0.0.2

TCP Header:
  Source Port: 45678
  Dest Port: 8080
  Sequence: 0x12345678
  Flags: SYN
  Window: 65535

Implementation Hints: Packet structures:

struct eth_header {
    uint8_t dest[6];
    uint8_t src[6];
    uint16_t type;  // 0x0800 = IP, 0x0806 = ARP
} __attribute__((packed));

struct ip_header {
    uint8_t version_ihl;
    uint8_t tos;
    uint16_t total_length;
    uint16_t id;
    uint16_t flags_fragment;
    uint8_t ttl;
    uint8_t protocol;  // 1 = ICMP, 6 = TCP, 17 = UDP
    uint16_t checksum;
    uint32_t src_ip;
    uint32_t dest_ip;
} __attribute__((packed));

Use TAP devices or raw sockets to send/receive Ethernet frames. Implement checksum calculation for IP and TCP headers.

Learning milestones:

ARP works → You understand MAC address resolution
ICMP ping works → You understand IP and network diagnostics
TCP handshake works → You understand reliable transport

Capstone Project: Write Your Own Operating System

File: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md
Main Programming Language: C + Assembly
Alternative Programming Languages: Rust + Assembly, C++ + Assembly
Coolness Level: Level 5: Pure Magic (Super Cool)
Business Potential: 1. The “Resume Gold” (Educational/Personal Brand)
Difficulty: Level 5: Master (The First-Principles Wizard)
Knowledge Area: Complete Operating System
Software or Tool: Custom OS
Main Book: “Operating Systems: Three Easy Pieces” by Arpaci-Dusseau

What you’ll build: A complete (minimal) operating system from scratch: bootloader, kernel, memory management, process management, file system, and shell. This is the ultimate test of OS understanding.

Why it teaches everything: This capstone integrates every concept: boot (Project 1-2), memory (Projects 3-4), processes (Projects 5-6), files (Project 9), IPC (Project 11), and the Unix philosophy (Projects 7, 10, 12). Building a complete OS means you truly understand how computers work.

Core challenges you’ll face:

Bootstrapping → maps to getting from power-on to kernel
Memory management → maps to heap, virtual memory, protection
Process management → maps to creation, scheduling, termination
File system → maps to persistent storage organization
User/kernel separation → maps to protection and system calls
Shell → maps to user interface

Key Concepts:

Everything from previous projects!
Integration: “Operating Systems: Three Easy Pieces” - Full book
OS Design Decisions: “The Design of the Unix Operating System” - Maurice Bach
Implementation Details: “Operating Systems Design and Implementation” - Tanenbaum

Difficulty: Master Time estimate: 3-6 months Prerequisites: All previous projects

Real world outcome:

$ qemu-system-x86_64 -drive format=raw,file=myos.img

=== MyOS v1.0 ===
Bootloader: Loaded kernel at 0x100000
Kernel: Initializing GDT, IDT...
Kernel: Initializing memory manager (128 MB detected)
Kernel: Initializing process scheduler
Kernel: Mounting root filesystem
Kernel: Starting init process (PID 1)
Init: Starting shell on /dev/tty0

myos$ uname -a
MyOS 1.0 x86_64 MyKernel

myos$ ls /
bin/
dev/
etc/
home/
proc/

myos$ cat /proc/meminfo
Total:     128 MB
Used:      12 MB
Free:      116 MB
Buffers:   4 MB

myos$ ps
PID   STATE     MEM     CMD
1     running   256K    /bin/init
2     running   512K    /bin/sh

myos$ echo "Hello from MyOS!" > /home/hello.txt
myos$ cat /home/hello.txt
Hello from MyOS!

myos$ # Pipes work!
myos$ ls /bin | wc -l
12

myos$ # Background processes work!
myos$ sleep 10 &
[1] 3
myos$ ps
PID   STATE     MEM     CMD
1     running   256K    /bin/init
2     running   512K    /bin/sh
3     sleeping  128K    /bin/sleep

myos$ # You built this. All of it.

Implementation Hints: Start with the OSDev wiki tutorials, but implement everything yourself:

Boot: Real mode → protected mode → long mode (64-bit)
Memory: Physical page allocator → virtual memory (paging) → heap allocator
Interrupts: Set up IDT, handle timer, keyboard
Processes: Task structure, context switch in assembly, simple scheduler
System calls: Software interrupt, dispatch table
File system: Start with ramfs, then persistent fs on disk
Shell: Parse commands, fork+exec, pipes

Milestones:

Week 1-2: Boot to protected mode, print to screen
Week 3-4: Interrupts, keyboard input
Week 5-6: Memory management
Week 7-10: Processes and scheduling
Week 11-14: File system
Week 15+: Shell, utilities, polish

Learning milestones:

Kernel boots and prints → You understand the boot process
Processes run and switch → You understand CPU virtualization
Memory is protected → You understand isolation
Files persist → You understand storage abstraction
Shell runs commands → You’ve built a complete OS!

Project Comparison Table

Project	Difficulty	Time	Unix Depth	Fun Factor	Practical Value
1. Bare Metal Hello	Expert	Weekend	⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐
2. Bootloader	Expert	1-2 weeks	⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐
3. Memory Allocator	Advanced	2 weeks	⭐⭐⭐	⭐⭐⭐	⭐⭐⭐⭐⭐
4. Virtual Memory Sim	Expert	2-3 weeks	⭐⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐
5. Scheduler Simulator	Advanced	2 weeks	⭐⭐⭐⭐	⭐⭐⭐	⭐⭐⭐
6. Fork and Exec	Intermediate	1 week	⭐⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐⭐
7. Unix Shell	Advanced	3-4 weeks	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐
8. System Call Interface	Expert	2 weeks	⭐⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐
9. File System	Expert	4-6 weeks	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐
10. Device Files	Advanced	2 weeks	⭐⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐
11. IPC Mechanisms	Advanced	2-3 weeks	⭐⭐⭐⭐	⭐⭐⭐	⭐⭐⭐⭐⭐
12. Unix Philosophy Tools	Intermediate	2-3 weeks	⭐⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐
13. Containers	Expert	3-4 weeks	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐
14. xv6 Study	Expert	4-8 weeks	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐
15. Kernel Module	Expert	3-4 weeks	⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐
16. Network Stack	Expert	4-6 weeks	⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐
Capstone: Your OS	Master	3-6 months	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐

Recommended Learning Path

Phase 1: Hardware-Software Boundary (4-6 weeks)

Goal: Understand what happens before the OS even loads

Bare Metal Hello World - See the raw hardware
Bootloader - Load your own kernel

Phase 2: Memory (4-6 weeks)

Goal: Understand how memory becomes usable

Memory Allocator - Build malloc/free
Virtual Memory Simulator - Understand paging

Phase 3: Processes (6-8 weeks)

Goal: Understand how programs run concurrently

Scheduler Simulator - Understand scheduling algorithms
Fork and Exec - The Unix process model
Unix Shell - Put it all together

Phase 4: The Unix Philosophy (4-6 weeks)

Goal: Understand why Unix works the way it does

Unix Philosophy Tools - Build cat, grep, sort
IPC Mechanisms - How processes communicate
Everything Is a File - Device files

Phase 5: Deep System (6-8 weeks)

Goal: Understand kernel internals

System Call Interface - The kernel boundary
File System - Persistent storage

Phase 6: Modern Unix (4-6 weeks)

Goal: Understand modern Linux

Containers - Namespaces and cgroups
Kernel Modules - Extend the kernel

Phase 7: Complete Understanding (8-12 weeks)

Goal: See it all work together

xv6 Study - Understand a complete OS
Network Stack - Understand networking

Phase 8: Mastery (3-6 months)

Goal: Prove you understand it all

Capstone: Your Own OS - Build it from scratch

Start Here Recommendation

Given your goal to understand OS fundamentals and the Unix philosophy:

Start with Project 6: Fork and Exec

Why?

It’s accessible—you can start immediately with just C knowledge
It reveals the heart of Unix’s elegance
You’ll understand why shells work the way they do
It builds directly into Project 7 (Shell)

Then do Project 12: Unix Philosophy Tools

Why?

Building cat, grep, sort shows why “one thing well” works
Composing them in pipelines is an “aha!” moment
You’ll see why text streams are powerful

After these two, you’ll have deep intuition for the Unix philosophy. Then tackle the lower-level projects (1-5) to understand what’s under the hood.

Why Unix Won: The Definitive Explanation

After completing these projects, you’ll understand this deeply:

1. SIMPLICITY COMPOUNDS
   - Simple tools compose into complex systems
   - Complex tools don't compose
   - 50 years later, `cat | grep | sort` still works

2. TEXT IS UNIVERSAL
   - Programs don't need to understand each other
   - Just read lines, process, write lines
   - JSON, YAML, config files—all text

3. FILES ARE EVERYTHING
   - One interface for devices, pipes, sockets, processes
   - Programs written for files work with everything
   - No special APIs for special cases

4. FORK+EXEC ENABLES SHELLS
   - Redirect after fork, before exec
   - No need for shell to understand programs
   - Any new program automatically works

5. WORSE IS BETTER
   - Simple, incomplete solution that ships
   - Beats perfect solution that doesn't
   - Unix shipped. Others didn't.

The internet, cloud, containers, DevOps—all built on Unix
Because when you bet on simplicity, you bet on the future.

Summary

#	Project Name	Main Language
1	Bare Metal “Hello World” (No OS)	Assembly (x86 NASM)
2	Bootloader that Loads a Kernel	Assembly + C
3	Memory Allocator (malloc from Scratch)	C
4	Virtual Memory Simulator	C
5	Process Scheduler Simulator	C
6	Fork and Exec (Process Creation)	C
7	Unix Shell from Scratch	C
8	System Call Interface	C + Assembly
9	File System from Scratch	C
10	Everything Is a File (Device Files)	C
11	Inter-Process Communication	C
12	Unix Philosophy Tools	C
13	Containers from Scratch	C
14	xv6 Operating System Study	C
15	Kernel Module Development	C
16	Network Stack Exploration	C
Capstone	Write Your Own Operating System	C + Assembly

“An operating system is the layer that says ‘you shall not’ to programs that want to do dangerous things, and ‘yes, here you go’ to programs that ask nicely through system calls. Understanding this boundary—between user and kernel, between process and process, between program and hardware—is understanding the computer itself.”