Operating Systems from First Principles - Expanded Project Guides

Generated from: OPERATING_SYSTEMS_FROM_FIRST_PRINCIPLES.md

Overview

This directory contains 17 comprehensive project guides that take you from bare metal to a complete operating system. Each project builds understanding of core OS concepts through hands-on implementation.

Philosophy: Operating systems are fundamentally about three things: processes, memory, and files. Everything else is implementation detail. These projects reveal how these abstractions are built and why Unix’s approach has proven so enduring.

Project Index

# Project Difficulty Time Key Focus
1 Bare Metal “Hello World” Expert Weekend Boot process, memory-mapped I/O
2 Bootloader that Loads a Kernel Expert 1-2 weeks Protected mode, GDT, disk I/O
3 Memory Allocator (malloc) Advanced 2 weeks Heap management, fragmentation
4 Virtual Memory Simulator Expert 2-3 weeks Paging, TLB, page replacement
5 Process Scheduler Simulator Advanced 2 weeks Scheduling algorithms, fairness
6 Fork and Exec (Process Creation) Intermediate 1 week Unix process model
7 Unix Shell from Scratch Advanced 3-4 weeks Pipes, redirection, job control
8 System Call Interface Expert 2 weeks Kernel boundary, syscall ABI
9 File System from Scratch Expert 4-6 weeks Inodes, blocks, directories
10 Everything Is a File (Device Files) Advanced 2 weeks Character devices, FUSE
11 Inter-Process Communication Advanced 2-3 weeks Pipes, shared memory, sockets
12 Unix Philosophy Tools Intermediate 2-3 weeks cat, grep, sort, pipelines
13 Containers from Scratch Expert 3-4 weeks Namespaces, cgroups
14 xv6 Operating System Study Expert 4-8 weeks Complete Unix-like OS
15 Kernel Module Development Expert 3-4 weeks Linux kernel programming
16 Network Stack Exploration Expert 4-6 weeks TCP/IP, raw sockets
17 Capstone: Write Your Own OS Master 3-6 months Complete operating system

Learning Paths

Path 1: Quick Understanding (6-8 weeks)

For developers who want to understand OS concepts quickly:

  1. P06 - Fork and Exec - The Unix process model
  2. P12 - Unix Philosophy Tools - Why “one thing well” works
  3. P07 - Unix Shell - Put it together
  4. P03 - Memory Allocator - What malloc does

Path 2: Deep Systems (3-4 months)

For those building systems careers:

  1. P01 - Bare Metal Hello
  2. P02 - Bootloader
  3. P03 - Memory Allocator
  4. P04 - Virtual Memory
  5. P06 - Fork and Exec
  6. P07 - Unix Shell
  7. P08 - System Calls
  8. P09 - File System

Path 3: Modern Linux (4-6 weeks)

For DevOps/SRE professionals:

  1. P13 - Containers - How Docker works
  2. P11 - IPC - Process communication
  3. P15 - Kernel Modules - Extend the kernel

Path 4: Complete Mastery (6-12 months)

For aspiring OS developers:

Complete all projects in order, culminating in:

Prerequisites

Essential

  • Strong C programming (pointers, memory, data structures)
  • Basic understanding of computer architecture
  • Linux/Unix command line proficiency
  • Willingness to read documentation and debug

Helpful

  • Assembly language basics (x86 or ARM)
  • Experience with GDB or LLDB
  • Understanding of networking concepts
  • Prior systems programming experience

The Core OS Abstractions

┌─────────────────────────────────────────────────────────────────┐
│                        USER PROGRAMS                            │
├─────────────────────────────────────────────────────────────────┤
│                     SYSTEM CALL INTERFACE                       │
├───────────────────┬───────────────────┬─────────────────────────┤
│     PROCESSES     │      MEMORY       │      FILES/IO           │
│   (P5, P6, P7)    │    (P3, P4)       │    (P9, P10)            │
│                   │                   │                         │
│  - Creation       │  - Allocation     │  - Everything is a file │
│  - Scheduling     │  - Virtual memory │  - Read/Write/Open/Close│
│  - Communication  │  - Protection     │  - Directories          │
│  - Termination    │  - Sharing        │  - Devices              │
├───────────────────┴───────────────────┴─────────────────────────┤
│                         HARDWARE                                │
│              CPU         Memory         Disk/Devices            │
│            (P1, P2)                      (P16)                  │
└─────────────────────────────────────────────────────────────────┘

Why Unix Won

After completing these projects, you’ll understand:

  1. Simplicity compounds - Simple tools compose into complex systems
  2. Text is universal - Programs don’t need to understand each other
  3. Files are everything - One interface for all I/O
  4. Fork+exec enables shells - Redirect after fork, before exec
  5. Worse is better - Simple solution that ships beats perfect one that doesn’t

Key Books

Book Primary Topics Projects
Operating Systems: Three Easy Pieces All core concepts All
The Linux Programming Interface System calls, IPC P6-P12
Computer Systems: A Programmer’s Perspective Memory, linking P1-P4, P8
Linux Device Drivers Kernel modules, devices P10, P15
Container Security Namespaces, cgroups P13
TCP/IP Illustrated, Volume 1 Networking P16
xv6 Book (MIT) Complete OS P14, P17

Getting Started

  1. Choose your path based on your goals and time
  2. Set up your environment:
    • Linux system (native or VM)
    • GCC, NASM, make, GDB
    • QEMU for bare-metal projects
  3. Start with Project 6 if you’re unsure - it’s accessible and reveals Unix’s elegance
  4. Build each project before looking at hints
  5. Trace through code - use GDB, add printfs, understand every line

“An operating system is the layer that says ‘you shall not’ to programs that want to do dangerous things, and ‘yes, here you go’ to programs that ask nicely through system calls.”