Project 3: Build and Boot Your First Custom Kernel

Build the Linux kernel from source, create a minimal root filesystem, and boot it in QEMU—with modifications you can observe.

Quick Reference

Attribute	Value
Difficulty	Intermediate
Time Estimate	Weekend
Language	C (kernel), Shell (scripting)
Prerequisites	Projects 1-2, understanding of makefiles
Key Topics	Kconfig, Kbuild, QEMU, boot process

---

1. Learning Objectives

By completing this project, you will:

• Understand the kernel build system (Kconfig and Kbuild)
• Learn the Linux boot process from firmware to userspace
• Establish a kernel development workflow with QEMU
• Make your first kernel modification and see the results

---

2. Theoretical Foundation

2.1 Core Concepts

Kconfig: The kernel’s configuration system. It defines thousands of options that control which features are built, as modules or built-in. The .config file stores your choices.

Kbuild: The kernel’s make-based build system. It compiles only what’s needed based on your configuration.

initramfs: An initial RAM filesystem that the kernel uses before mounting the real root filesystem. Contains essential tools needed for early boot.

Boot Process Overview:
┌─────────────────────────────────────────────────────────────┐
│                      Power On                                │
│                         │                                    │
│                         ▼                                    │
│  ┌─────────────────────────────────────────────────────┐    │
│  │                 Firmware (BIOS/UEFI)                 │    │
│  │  - POST (hardware initialization)                    │    │
│  │  - Find bootable device                              │    │
│  │  - Load bootloader                                   │    │
│  └──────────────────────┬──────────────────────────────┘    │
│                         ▼                                    │
│  ┌─────────────────────────────────────────────────────┐    │
│  │                 Bootloader (GRUB)                    │    │
│  │  - Load kernel (bzImage) into memory                 │    │
│  │  - Load initramfs into memory                        │    │
│  │  - Jump to kernel entry point                        │    │
│  └──────────────────────┬──────────────────────────────┘    │
│                         ▼                                    │
│  ┌─────────────────────────────────────────────────────┐    │
│  │                    Kernel                            │    │
│  │  - Decompress itself                                 │    │
│  │  - Initialize hardware, memory, scheduler           │    │
│  │  - Mount initramfs as root                          │    │
│  │  - Execute /init from initramfs                     │    │
│  └──────────────────────┬──────────────────────────────┘    │
│                         ▼                                    │
│  ┌─────────────────────────────────────────────────────┐    │
│  │                  /init (initramfs)                   │    │
│  │  - Load necessary drivers                           │    │
│  │  - Mount real root filesystem                       │    │
│  │  - pivot_root or switch_root to real root           │    │
│  └──────────────────────┬──────────────────────────────┘    │
│                         ▼                                    │
│  ┌─────────────────────────────────────────────────────┐    │
│  │                  Init System (systemd)              │    │
│  │  - Start services                                   │    │
│  │  - Reach multi-user target                          │    │
│  └─────────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────┘

2.2 Why This Matters

You cannot contribute to the kernel without being able to build and test it. This project establishes your development workflow:

• Fast iteration: Make a change, rebuild, reboot in seconds with QEMU
• Safe experimentation: Crash your VM, not your machine
• Debugging capability: GDB integration for stepping through kernel code

2.3 Historical Context

The kernel build system has evolved significantly:

• Early Linux: Simple Makefile
• 2.4: Introduced make menuconfig
• 2.6: Kconfig/Kbuild system introduced
• Modern: Supports thousands of options, cross-compilation, out-of-tree builds

2.4 Common Misconceptions

• “You need a powerful machine” - A modest machine can build the kernel in ~20 minutes; ccache makes rebuilds much faster.
• “You need to install the kernel” - For development, boot directly in QEMU without touching your host.
• “Every change needs a full rebuild” - Incremental builds only recompile changed files.

---

3. Project Specification

3.1 What You Will Build

A complete kernel development workflow:

1. Build scripts for the kernel
2. A minimal initramfs with busybox
3. QEMU launch scripts with GDB support
4. Your first kernel modification (custom version string)

3.2 Functional Requirements

1. Build System:
   • Script to configure kernel for QEMU
   • Script to build kernel with ccache
   • Clean rebuild option
2. Root Filesystem:
   • Minimal initramfs with BusyBox
   • Basic shell environment
   • Common utilities (ls, cat, mount, etc.)
3. QEMU Integration:
   • Boot script with serial console
   • GDB debugging support
   • Shared folder with host (9p)
4. Kernel Modification:
   • Custom EXTRAVERSION string
   • Custom boot message

3.3 Non-Functional Requirements

• Build time under 30 minutes (first build)
• Incremental rebuild under 2 minutes
• Boot time under 5 seconds in QEMU

3.4 Example Usage / Output

$ ./build_kernel.sh
Configuring kernel for QEMU...
Building kernel (this takes a few minutes)...
  CC      init/main.o
  ...
Kernel ready at: bzImage

$ ./build_initramfs.sh
Creating minimal initramfs with BusyBox...
Initramfs ready at: initramfs.cpio.gz

$ ./boot_qemu.sh
[    0.000000] Linux version 6.8.0-MYKERNEL-learning (you@yourhost)
[    0.000000] Command line: console=ttyS0 nokaslr
[    0.012345] Hello from my modified kernel startup!
...
[    0.234567] Run /init as init process

Welcome to your custom kernel!
Kernel version: 6.8.0-MYKERNEL-learning
# uname -a
Linux (none) 6.8.0-MYKERNEL-learning #1 SMP PREEMPT_DYNAMIC ...

# cat /proc/version
Linux version 6.8.0-MYKERNEL-learning (you@yourhost) ...

# dmesg | grep -i "hello"
[    0.001234] Hello from my modified kernel startup!

3.5 Real World Outcome

You’ll have a working kernel development environment that you’ll use for all subsequent projects. This is the same workflow used by kernel developers worldwide.

---

4. Solution Architecture

4.1 High-Level Design

┌─────────────────────────────────────────────────────────────┐
│                    Development Host                          │
│                                                              │
│  ┌──────────────┐   ┌──────────────┐   ┌──────────────┐    │
│  │   Kernel     │   │  Initramfs   │   │    QEMU      │    │
│  │   Source     │   │   Builder    │   │   Launcher   │    │
│  │    Tree      │   │              │   │              │    │
│  └──────┬───────┘   └──────┬───────┘   └──────┬───────┘    │
│         │                  │                  │             │
│    make bzImage      cpio archive          qemu-system     │
│         │                  │                  │             │
│         ▼                  ▼                  ▼             │
│  ┌──────────────────────────────────────────────────┐      │
│  │                    Artifacts                      │      │
│  │  - bzImage (kernel)                               │      │
│  │  - initramfs.cpio.gz                             │      │
│  │  - vmlinux (for GDB symbols)                     │      │
│  └──────────────────────────────────────────────────┘      │
│                                                              │
└─────────────────────────────────────────────────────────────┘
                              │
                              │ qemu-system-x86_64
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                      QEMU Virtual Machine                    │
│  ┌──────────────────────────────────────────────────┐      │
│  │               Your Custom Kernel                  │      │
│  │  + initramfs                                      │      │
│  │  + BusyBox shell                                  │      │
│  └──────────────────────────────────────────────────┘      │
│                              ▲                               │
│                              │ GDB (port 1234)              │
└─────────────────────────────────────────────────────────────┘

4.2 Key Components

1. Kernel Source: The Linux kernel git repository
2. Build Scripts: Configuration and compilation automation
3. Initramfs: Minimal root filesystem for booting
4. QEMU Scripts: VM launch with various options

4.3 Directory Structure

kernel-dev/
├── linux/                    # Kernel source tree
│   └── .config              # Your kernel config
├── initramfs/               # Initramfs contents
│   ├── init                 # Init script
│   ├── bin/                 # BusyBox symlinks
│   └── ...
├── scripts/
│   ├── build_kernel.sh      # Build the kernel
│   ├── build_initramfs.sh   # Create initramfs
│   ├── boot_qemu.sh         # Launch QEMU
│   └── debug_kernel.sh      # Launch with GDB
├── artifacts/
│   ├── bzImage              # Compiled kernel
│   ├── vmlinux              # Kernel with symbols
│   └── initramfs.cpio.gz    # Compressed initramfs
└── README.md

4.4 Algorithm Overview

Build Process:

1. Configure: make defconfig + customizations
2. Build: make -j$(nproc) bzImage
3. Result: arch/x86/boot/bzImage

Initramfs Creation:

1. Create directory structure
2. Install BusyBox (static)
3. Create init script
4. Pack: find . | cpio -o -H newc | gzip > initramfs.cpio.gz

Boot Process:

1. QEMU loads kernel and initramfs into VM memory
2. Kernel decompresses, initializes
3. Kernel extracts initramfs to tmpfs
4. Kernel executes /init
5. Shell prompt appears

---

5. Implementation Guide

5.1 Development Environment Setup

# Install required tools
sudo apt install build-essential libncurses-dev bison flex \
    libssl-dev libelf-dev bc dwarves git ccache qemu-system-x86

# Get kernel source
git clone --depth 1 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

# Set up ccache for faster rebuilds
export PATH="/usr/lib/ccache:$PATH"

5.2 Project Structure

Create the workspace:

mkdir -p kernel-dev/{scripts,initramfs/{bin,sbin,etc,proc,sys,dev,tmp},artifacts}
cd kernel-dev

5.3 The Core Question You’re Answering

“How does the Linux kernel boot, and how can I efficiently iterate on kernel changes?”

5.4 Concepts You Must Understand First

1. What is a kernel image format?
   • What’s the difference between vmlinux, bzImage, and vmlinuz?
   • Reference: Kernel documentation on boot
2. What is initramfs?
   • Why do we need an initial root filesystem?
   • How does the kernel find and execute /init?
   • Reference: “How Linux Works” Chapter 5
3. What is QEMU?
   • How does hardware virtualization work?
   • What is KVM and why does it make QEMU fast?
   • Reference: QEMU documentation

5.5 Questions to Guide Your Design

Configuration:

• What kernel options are needed for QEMU boot?
• What drivers should be built-in vs modules?

Initramfs:

• What’s the minimum needed to get a shell?
• How do you create device nodes?

Debugging:

• How do you connect GDB to the kernel?
• What is KASLR and why disable it for debugging?

5.6 Thinking Exercise

Before running any commands, trace the boot process:

1. QEMU starts, BIOS/UEFI equivalent runs
2. QEMU’s direct kernel boot loads bzImage at 0x100000
3. Kernel’s setup code runs in real mode
4. Kernel switches to protected mode, then long mode
5. Kernel decompresses itself
6. start_kernel() in init/main.c runs
7. Kernel initializes all subsystems
8. Kernel mounts initramfs from memory
9. Kernel executes /init
10. Your shell prompt appears

Question: What would happen if /init doesn’t exist in your initramfs?

5.7 Hints in Layers

Hint 1 - Basic Configuration:

cd linux
make defconfig                    # Start with defaults
make kvm_guest.config            # Add KVM options
scripts/config --enable DEBUG_INFO  # Enable debug symbols

Hint 2 - Essential Config Options:

# For QEMU boot
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_BLK=y
CONFIG_VIRTIO_NET=y
CONFIG_9P_FS=y                   # Share files with host

# For debugging
CONFIG_DEBUG_INFO=y
CONFIG_GDB_SCRIPTS=y
CONFIG_DEBUG_KERNEL=y

# For serial console
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y

Hint 3 - Minimal Init Script:

#!/bin/sh
# /init for initramfs

# Mount essential filesystems
mount -t proc none /proc
mount -t sysfs none /sys
mount -t devtmpfs none /dev

# Print welcome
echo "Welcome to your custom kernel!"
echo "Kernel version: $(uname -r)"

# Launch shell
exec /bin/sh

Hint 4 - QEMU Command:

qemu-system-x86_64 \
    -kernel linux/arch/x86/boot/bzImage \
    -initrd artifacts/initramfs.cpio.gz \
    -append "console=ttyS0 nokaslr" \
    -nographic \
    -enable-kvm \
    -m 1G \
    -s                          # GDB stub on port 1234

5.8 The Interview Questions They’ll Ask

1. “Explain the difference between bzImage, vmlinux, and vmlinuz”
   • vmlinux: Raw kernel ELF, with debug symbols
   • bzImage: Compressed kernel + bootloader setup code
   • vmlinuz: Usually a symlink to bzImage
2. “What is the kernel command line and how is it used?”
   • Passed by bootloader to kernel
   • Controls boot behavior (root=, console=, init=, etc.)
   • Available in /proc/cmdline
3. “Why would you use initramfs instead of direct root mount?”
   • Root filesystem might need drivers loaded first
   • Encrypted root needs unlock before mount
   • Network boot needs network stack first
4. “What is CONFIG_LOCALVERSION?”
   • Appended to kernel version string
   • Helps identify custom builds
   • Shows in uname -r
5. “How do you debug a kernel crash?”
   • Serial console output
   • GDB attached to QEMU
   • crash dump analysis with crash utility

5.9 Books That Will Help

Topic	Book	Chapter
Kernel build system	Linux Kernel Programming	Chapters 2-3
Boot process	How Linux Works	Chapter 5
Kernel configuration	Linux Device Drivers, 3rd Ed	Chapter 2
QEMU/KVM	Various QEMU docs	-

5.10 Implementation Phases

Phase 1: Build the Kernel (Day 1)

• Clone kernel source
• Configure with make defconfig
• Build with make -j$(nproc) bzImage
• Note the location of bzImage

Phase 2: Create Initramfs (Day 1)

• Download/build static BusyBox
• Create directory structure
• Write init script
• Create cpio archive

Phase 3: Boot in QEMU (Day 1)

• Write QEMU launch script
• Boot and verify shell works
• Test basic commands

Phase 4: Kernel Modifications (Day 2)

• Change EXTRAVERSION in Makefile
• Add printk to start_kernel()
• Rebuild and verify changes

Phase 5: GDB Integration (Day 2)

• Add debug config options
• Launch QEMU with -s -S
• Connect GDB, set breakpoint, continue

5.11 Key Implementation Decisions

Decision 1: What kernel version to use

• Latest stable is fine for learning
• Avoid release candidates (unstable)

Decision 2: How to configure

• Start with defconfig (reasonable defaults)
• Add kvm_guest.config for QEMU
• Manually enable debug options

Decision 3: initramfs vs actual rootfs

• initramfs is simpler for development
• No disk image to manage
• Everything in RAM, fast boot

---

6. Testing Strategy

Build Tests

# Verify kernel builds successfully
make -j$(nproc) bzImage
[ -f arch/x86/boot/bzImage ] && echo "Kernel build OK"

# Verify vmlinux has symbols (for GDB)
file vmlinux | grep -q "not stripped" && echo "Symbols OK"

Boot Tests

# Test basic boot (timeout if hangs)
timeout 30 qemu-system-x86_64 \
    -kernel bzImage -initrd initramfs.cpio.gz \
    -append "console=ttyS0" -nographic \
    | grep -q "Welcome" && echo "Boot OK"

Modification Tests

# Verify custom version shows
qemu-system-x86_64 ... -append "..." | grep "MYKERNEL"

# Verify custom printk shows
qemu-system-x86_64 ... | grep "Hello from my modified"

---

7. Common Pitfalls & Debugging

Problem	Symptom	Root Cause	Fix
Build fails	“elf.h not found”	Missing dev packages	apt install libelf-dev
QEMU hangs	Black screen	Missing serial console config	Add console=ttyS0 to cmdline
No shell prompt	Kernel panic	No /init in initramfs	Check initramfs creation
GDB won’t connect	Connection refused	QEMU not started with -s	Add -s flag
Slow boot	Takes 30+ seconds	KVM not enabled	Add -enable-kvm, check /dev/kvm

Quick Verification

# Check KVM is available
ls -la /dev/kvm

# Check kernel config is correct
grep CONFIG_SERIAL_8250_CONSOLE linux/.config

# Check initramfs has /init
gunzip -c initramfs.cpio.gz | cpio -t | grep "^init$"

---

8. Extensions & Challenges

Easy Extensions

1. Add more utilities: Include strace, gdb in initramfs
2. Add shared folder: 9p filesystem for host-guest file sharing
3. Multiple consoles: Add virtio-console support

Medium Extensions

1. Real root filesystem: Create ext4 image, mount after initramfs
2. Network support: Add virtio-net, configure networking
3. Module loading: Build some drivers as modules, load dynamically

Hard Extensions

1. Cross-compilation: Build for ARM, boot in QEMU ARM
2. Custom scheduler: Modify CFS, measure impact
3. Kernel live patching: Test kpatch/livepatch

---

9. Real-World Connections

How Kernel Developers Work

This is the actual workflow:

1. Clone kernel, configure, build
2. Boot in QEMU for fast iteration
3. Once stable, test on real hardware
4. Submit patches for review

Continuous Integration

Kernel CI systems like:

• KernelCI: Automated build and boot testing
• Intel 0-day: Tests every patch for regressions
• syzkaller: Fuzzing for bugs

They all use similar QEMU-based testing.

---

10. Resources

Essential Documentation

• Documentation/admin-guide/kernel-parameters.txt
• Documentation/kbuild/kconfig.rst
• Documentation/dev-tools/gdb-kernel-debugging.rst

Code References

• init/main.c - Kernel initialization
• arch/x86/boot/ - x86 boot code
• usr/ - initramfs building

Online Resources

• Kernel Newbies
• QEMU Documentation
• LWN: Speeding up kernel development with QEMU

---

11. Self-Assessment Checklist

Before moving to the next project, verify:

• I can build the kernel from source
• I understand what Kconfig options control
• I can create a bootable initramfs
• My kernel boots in QEMU with serial console
• I’ve modified the kernel and seen my changes
• I can attach GDB to the running kernel
• I understand the boot process from QEMU to shell
• I know what nokaslr does and why we use it

---

12. Submission / Completion Criteria

Your project is complete when:

1. Kernel builds without errors
2. Custom version string appears in uname -r
3. Custom boot message appears in dmesg
4. Shell is usable (ls, cat, mount work)
5. GDB can connect and hit breakpoints
6. Scripts are reusable and documented

Verification Commands

# These should all work:
./scripts/build_kernel.sh
./scripts/build_initramfs.sh
./scripts/boot_qemu.sh
# Inside QEMU:
uname -r | grep MYKERNEL
dmesg | grep "Hello from my"

---

Next Project: P04 - Kernel Module - Write loadable kernel modules to learn kernel programming fundamentals.