Project 16: Operating System Kernel

Build a minimal OS kernel that boots, manages memory, schedules tasks, and runs a shell.

Quick Reference

Attribute Value
Difficulty Master
Time Estimate 3-6 months
Language C + Assembly
Prerequisites Projects 1-15, OS concepts
Key Topics Boot, memory, scheduling, syscalls

1. Learning Objectives

By completing this project, you will:

  1. Boot a kernel on x86 or ARM using a bootloader.
  2. Implement physical and virtual memory management.
  3. Create a process scheduler and context switcher.
  4. Build a basic file system and shell.

2. Theoretical Foundation

2.1 Core Concepts

  • Bootstrapping: From reset to kernel entry point.
  • Virtual memory: Address translation and page tables.
  • Scheduling: Time-sliced multitasking.
  • System calls: Controlled kernel entry.

2.2 Why This Matters

An OS kernel integrates every systems concept: hardware interaction, memory management, process control, and I/O. Completing it demonstrates elite systems mastery.

2.3 Historical Context / Background

Operating systems evolved from simple monitors to complex kernels. Building one from scratch reveals the layered architecture of modern OSes.

2.4 Common Misconceptions

  • “The kernel is one big loop”: It is a set of coordinated subsystems.
  • “You can skip assembly”: Bootstrapping requires it.

3. Project Specification

3.1 What You Will Build

A minimal kernel that:

  • Boots in QEMU
  • Sets up paging
  • Runs multiple processes
  • Handles interrupts
  • Provides a small shell

3.2 Functional Requirements

  1. Bootloader loads kernel and transfers control.
  2. Initialize GDT/IDT (or equivalent).
  3. Memory allocator for kernel and user space.
  4. Process scheduler and context switching.
  5. Basic filesystem with read/write.

3.3 Non-Functional Requirements

  • Stability: No triple faults.
  • Debuggability: Serial logging and symbols.
  • Modularity: Clean separation of subsystems.

3.4 Example Usage / Output

$ qemu-system-x86_64 -kernel kernel.bin -serial stdio
[boot] kernel start
[mem] paging enabled
[sched] running init
shell> ls
bin  dev  etc
shell> run demo
[demo] hello from user process

3.5 Real World Outcome

You boot into your own kernel, run a basic shell, and execute user programs. This is a complete OS prototype running on real hardware emulation.

4. Solution Architecture

4.1 High-Level Design

bootloader -> kernel init -> memory -> scheduler -> filesystem -> shell

4.2 Key Components

Component Responsibility Key Decisions
Bootloader Load kernel Multiboot or custom
Memory manager Paging + alloc Buddy or bitmap
Scheduler Task switching Round-robin
Syscalls User-kernel API Interrupt-based
FS Store files Simple FAT-like

4.3 Data Structures

typedef struct {
    uint32_t pid;
    uint32_t *page_dir;
    struct Context ctx;
} Process;

4.4 Algorithm Overview

Key Algorithm: Context switch

  1. Save registers of current task.
  2. Load registers of next task.
  3. Switch page directory.
  4. Resume execution.

Complexity Analysis:

  • Scheduling: O(1) for round-robin
  • Syscalls: O(1) per trap

5. Implementation Guide

5.1 Development Environment Setup

# Cross-compiler recommended
x86_64-elf-gcc --version
qemu-system-x86_64 --version

5.2 Project Structure

kernel/
├── boot/
│   └── boot.s
├── kernel/
│   ├── main.c
│   ├── mm.c
│   ├── sched.c
│   └── syscall.c
├── fs/
│   └── fs.c
└── README.md

5.3 The Core Question You’re Answering

“How does an operating system manage hardware and provide safe abstractions to user programs?”

5.4 Concepts You Must Understand First

Stop and research these before coding:

  1. Boot process
    • From BIOS/UEFI to kernel entry.
  2. Paging
    • How virtual addresses map to physical.
  3. Interrupts
    • How the CPU transfers control to the kernel.

5.5 Questions to Guide Your Design

Before implementing, think through these:

  1. Will you target x86_64 or ARM?
  2. What bootloader standard will you use?
  3. What minimal filesystem features are required?

5.6 Thinking Exercise

Task Switch

If a timer interrupt fires, what state must you save to resume a task later?

5.7 The Interview Questions They’ll Ask

Prepare to answer these:

  1. “How does virtual memory work?”
  2. “What is a context switch?”
  3. “How do syscalls transfer control?”

5.8 Hints in Layers

Hint 1: Start with boot and serial output Print text to confirm boot.

Hint 2: Add memory allocator Implement a simple page allocator.

Hint 3: Add a scheduler Switch between two demo tasks first.

5.9 Books That Will Help

Topic Book Chapter
OS fundamentals “Operating Systems: Three Easy Pieces” Ch. 4-8
OS design “Operating System Concepts” Memory and scheduling

5.10 Implementation Phases

Phase 1: Boot and Output (3-4 weeks)

Goals:

  • Boot kernel and print output

Tasks:

  1. Write bootloader and kernel entry.
  2. Set up serial or VGA output.

Checkpoint: Kernel prints to screen/serial.

Phase 2: Memory and Interrupts (4-6 weeks)

Goals:

  • Paging and interrupts

Tasks:

  1. Enable paging.
  2. Set up IDT and timer interrupt.

Checkpoint: Timer interrupt fires reliably.

Phase 3: Scheduling and Syscalls (4-6 weeks)

Goals:

  • Multitasking and syscall interface

Tasks:

  1. Implement context switching.
  2. Add syscall handler.

Checkpoint: Two tasks run and yield.

Phase 4: Filesystem and Shell (4-6 weeks)

Goals:

  • Basic storage and user interface

Tasks:

  1. Implement simple filesystem.
  2. Add shell with built-in commands.

Checkpoint: Shell can read files.

5.11 Key Implementation Decisions

Decision Options Recommendation Rationale
Target arch x86_64 vs ARM x86_64 Better tooling
FS type Custom vs FAT Custom simple Smaller scope

6. Testing Strategy

6.1 Test Categories

Category Purpose Examples
Boot Tests Kernel loads QEMU boot
Memory Tests Paging Allocate/free pages
Scheduler Tests Task switching Two tasks toggling

6.2 Critical Test Cases

  1. Boot: Kernel entry reached.
  2. Paging: Access mapped memory.
  3. Syscalls: User program prints output.

6.3 Test Data

Demo tasks and simple user programs

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

Pitfall Symptom Solution
Triple fault VM resets Check IDT/GDT setup
Bad paging Page faults Validate page tables
Stack corruption Random crashes Separate kernel/user stacks

7.2 Debugging Strategies

  • Use QEMU with -s -S and GDB.
  • Add serial logging for early boot.

7.3 Performance Traps

Premature optimization can slow progress. Focus on correctness and visibility first.

8. Extensions & Challenges

8.1 Beginner Extensions

  • Add keyboard input driver.
  • Add simple heap allocator for user space.

8.2 Intermediate Extensions

  • Implement a basic TCP/IP stack.
  • Add ELF loader for user programs.

8.3 Advanced Extensions

  • Add virtual filesystem layer.
  • Implement SMP scheduling.

9. Real-World Connections

9.1 Industry Applications

  • Kernel engineering: Linux, BSD, embedded OSes.
  • Hypervisors: Many concepts carry over.
  • xv6: Teaching OS from MIT.
  • OSDev Wiki: Practical guides.

9.3 Interview Relevance

OS kernel knowledge is a top-tier signal for systems roles.

10. Resources

10.1 Essential Reading

  • “Operating Systems: Three Easy Pieces” - Ch. 4-8
  • xv6 source code
  • OSDev Wiki

10.2 Video Resources

  • OS development lecture series

10.3 Tools & Documentation

  • QEMU docs
  • GDB remote debugging
  • Bare Metal LED: Hardware bring-up skills.
  • Unix Shell: User interface component.

11. Self-Assessment Checklist

11.1 Understanding

  • I can explain the boot process.
  • I understand paging and interrupts.
  • I can describe context switching.

11.2 Implementation

  • Kernel boots in QEMU.
  • Scheduler runs multiple tasks.
  • Shell works for basic commands.

11.3 Growth

  • I can add new drivers.
  • I can explain this project in an interview.

12. Submission / Completion Criteria

Minimum Viable Completion:

  • Kernel boots and prints output.

Full Completion:

  • Paging, scheduler, and basic syscalls.

Excellence (Going Above & Beyond):

  • Filesystem and user programs running.

This guide was generated from C_PROGRAMMING_COMPLETE_MASTERY.md. For the complete learning path, see the parent directory.