LEARN CLOUD FROM SCRATCH IN C

Learn Cloud Infrastructure from Scratch in C

Goal: To deeply understand how modern cloud platforms like OpenShift/Kubernetes (Container-as-a-Service) and OpenStack (Infrastructure-as-a-Service) work by building their fundamental components from first principles in C on Linux.

Why Learn This Way?

OpenShift and OpenStack are two of the most complex open-source projects in existence. You can spend years learning their APIs and configurations without ever understanding why they work. This guide flips the script. Instead of learning the top-level abstractions, you will start at the bottom, with the Linux kernel syscalls, and build your way up.

This path is challenging but incredibly rewarding. By building a container runtime, a VM controller, and a network manager in C, you are not just learning “about” the cloud; you are learning the deep systems-level truths that make the cloud possible. You will understand what a container is, not just how to run one.

After completing these projects, you will:

Understand that a container is just a specially isolated Linux process.
Know how cgroups and namespaces, the pillars of containerization, work at the syscall level.
Be able to programmatically control virtual machines using a hypervisor API.
Understand how virtual networking is constructed from bridges and virtual network devices.
Grasp the core client-server architecture of a cloud control plane.

Core Concept Analysis

Modern cloud platforms, while vastly complex, are built from a handful of powerful Linux primitives.

The Anatomy of a Container (OpenShift/Kubernetes)

A container is not a “lightweight VM”. It is a standard Linux process given a restricted view of the system.

┌──────────────────────────────────────────────────┐
│              A SINGLE LINUX PROCESS              │
│                                                  │
│  ┌────────────┐   ┌────────────┐   ┌────────────┐│
│  │ Namespaces │   │   Cgroups  │   │  chroot /  ││
│  │ Isolate     │   │ Limit      │   │ Pivot Root ││
│  │ What it SEES│   │ What it USES│   │ Change FS  ││
│  └────────────┘   └────────────┘   └────────────┘│
│       │                 │                │       │
│       ▼                 ▼                ▼       │
│  "I'm the only  │  "I can only │  "This folder is │
│  process (PID 1)"│ use 100MB RAM"│  my whole world" │
│  "I have my own │               │                │
│  network stack" │               │                │
└──────────────────────────────────────────────────┘
                         │
                         ▼
┌──────────────────────────────────────────────────┐
│                  "THE CONTAINER"                 │
└──────────────────────────────────────────────────┘

The Anatomy of an IaaS Cloud (OpenStack)

An IaaS cloud automates the lifecycle of virtual machines, abstracting away the underlying hardware.

┌────────────────────┐
│      User / API    │
└──────────┬─────────┘
           │ HTTP Request ("Create VM")
           ▼
┌────────────────────┐
│    Control Plane   │
│   (API, Scheduler) │
└──────────┬─────────┘
           │ Command ("Put this VM on Host 2")
           ▼
┌────────────────────────────────────────────────────────────────┐
│                           DATA PLANE                           │
│                                                                │
│ ┌────────────┐   ┌────────────┐   ┌────────────┐   ┌────────────┐
│ │ Hypervisor │   │ Hypervisor │   │  Network   │   │  Storage   │
│ │   (Host 1)   │   │   (Host 2)   │   │  (Virtual  │   │  (Network  │
│ │            │   │            │   │   Switches)  │   │   Disks)   │
│ └────────────┘   └────────────┘   └────────────┘   └────────────┘
└────────────────────────────────────────────────────────────────┘

Project List: The Path from Process to Cloud

This path is divided into two parts. Part 1 builds the primitives of containers. Part 2 builds the primitives of IaaS and VMs.

Part 1: Building a Container Runtime from Scratch

Project 1: The `chroot` Jail

File: LEARN_CLOUD_FROM_SCRATCH_IN_C.md
Main Programming Language: C
Alternative Programming Languages: Go, Rust
Coolness Level: Level 3: Genuinely Clever
Business Potential: 1. The “Resume Gold”
Difficulty: Level 2: Intermediate
Knowledge Area: Operating Systems / System Calls
Software or Tool: Linux Kernel Syscalls (chroot, fork, execvp)
Main Book: “The Linux Programming Interface” by Michael Kerrisk

What you’ll build: A C program that takes a directory and a command as arguments (e.g., ./myjail /tmp/rootfs /bin/sh), and executes that command “jailed” inside the given directory, making the parent filesystem inaccessible.

Why it teaches the fundamentals: This is the first step in process isolation. You learn that a process’s view of the filesystem is malleable. This is the oldest and simplest form of “containerization,” and the direct ancestor of Docker’s filesystem layers.

Core challenges you’ll face:

Calling the chroot syscall → maps to understanding its security implications and requirements (requires root)
Managing processes with fork and execvp → maps to standard UNIX process control
Setting up a minimal root filesystem → maps to copying necessary executables (/bin/sh, /bin/ls) and their shared libraries (ldd) into your jail directory
Waiting for the child process → maps to using waitpid to properly clean up child processes

Key Concepts:

chroot(2): The man page is your primary resource.
Process Lifecycle: “The Linux Programming Interface”, Chapters 24-26 - Kerrisk
Shared Libraries: Understanding the output of the ldd command.

Difficulty: Intermediate Time estimate: Weekend Prerequisites: Solid C programming skills, comfort with the Linux command line.

Real world outcome: You’ll have a program that creates a sandbox. When you run /bin/sh inside it, an ls / will only show the contents of your jail directory, not your real root filesystem.

# Setup a minimal filesystem
$ mkdir -p /tmp/rootfs/bin
$ cp /bin/ls /bin/sh /tmp/rootfs/bin/
# ... plus dependencies found with `ldd` ...

# Run your program
$ sudo ./myjail /tmp/rootfs /bin/sh
# (Inside the new shell)
$ ls /
bin
$ ls /bin
ls sh
$ exit

Learning milestones:

Your program successfully executes a command inside the chroot → You understand basic process jailing.
You can create a functional rootfs with necessary libraries → You understand runtime dependencies.
You understand why chroot alone is not secure → You’re ready for the next step: namespaces.

Project 2: Real Isolation with Namespaces

File: LEARN_CLOUD_FROM_SCRATCH_IN_C.md
Main Programming Language: C
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 1. The “Resume Gold”
Difficulty: Level 3: Advanced
Knowledge Area: OS Internals / Linux Kernel
Software or Tool: Linux clone(2) syscall
Main Book: “The Linux Programming Interface” by Michael Kerrisk

What you’ll build: A C program that uses the clone() syscall with the CLONE_NEWPID and CLONE_NEWUTS flags. The child process will run /bin/bash and will have its own separate process ID namespace (it will see itself as PID 1) and its own hostname.

Why it teaches the fundamentals: This is the core magic of modern containers. You’ll learn that a container is just a process whose “view” of the system’s resources (like the PID table or hostname) has been virtualized by the kernel. This is the heart of what tools like Docker and runc do.

Core challenges you’ll face:

Using the clone syscall instead of fork → maps to understanding how to create a child process with special properties
Providing a separate stack for the child → maps to a tricky requirement of clone() that forces you to manage memory manually
Passing the correct namespace flags → maps to CLONE_NEWPID, CLONE_NEWUTS, CLONE_NEWNS
Understanding the implications of PID namespaces → maps to realizing why the child process sees itself as PID 1 and cannot see processes outside its namespace

Key Concepts:

Namespaces: “The Linux Programming Interface”, Chapter 38 - Kerrisk
clone(2) syscall: Man page is essential. Note the function signature is complex.
Stack Allocation: Using malloc and being careful about the direction of stack growth.

Difficulty: Advanced Time estimate: Weekend Prerequisites: Project 1, deep understanding of C pointers and memory allocation.

Real world outcome: You’ll run your C program. It will spawn a bash shell. Inside that shell, running ps aux will only show the bash process itself (as PID 1) and its children. Running hostname will show a different name from your host machine.

// Conceptual C code
int child_func(void *arg) {
  // Set a new hostname
  sethostname("container", 10);
  // Execute a shell
  execvp("/bin/bash", (char *[]){ "/bin/bash", NULL });
  return 0;
}

int main() {
  // Allocate a stack for the child
  char *stack = malloc(STACK_SIZE);
  char *stackTop = stack + STACK_SIZE;

  // Call clone with namespace flags
  clone(child_func, stackTop, CLONE_NEWUTS | CLONE_NEWPID | SIGCHLD, NULL);

  wait(NULL);
  return 0;
}

Learning milestones:

Your child process starts and runs a shell → You have successfully used clone().
ps aux inside the shell shows the shell as PID 1 → You have created a PID namespace.
hostname inside the shell is different from your host’s → You have created a UTS namespace.
You feel a profound sense of understanding: a container is just a process → The magic is gone, replaced by knowledge.

Project 3: Resource Limiting with Cgroups

File: LEARN_CLOUD_FROM_SCRATCH_IN_C.md
Main Programming Language: C
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 1. The “Resume Gold”
Difficulty: Level 3: Advanced
Knowledge Area: OS Internals / Linux Kernel
Software or Tool: Cgroups V2 filesystem API
Main Book: “How Linux Works, 3rd Edition” by Brian Ward

What you’ll build: A C program that creates a cgroup, sets a memory limit (e.g., 50MB), adds the current process to that cgroup, and then forks a child process that tries to allocate more memory than the limit, causing it to be killed by the kernel’s Out-of-Memory (OOM) killer.

Why it teaches the fundamentals: This is the second pillar of containerization. Namespaces isolate what a process can see; cgroups isolate what a process can use. You will learn that resource limits aren’t a feature of Docker, but a fundamental feature of the Linux kernel, exposed through a special filesystem.

Core challenges you’ll face:

Interacting with the cgroup filesystem → maps to creating directories and writing to special files in /sys/fs/cgroup
Setting a memory limit → maps to writing a value like 50M to the memory.max file
Adding a process to a cgroup → maps to writing a PID to the cgroup.procs file
Observing the OOM killer in action → maps to seeing your memory-hungry child process get terminated by the kernel

Key Concepts:

Cgroups v2: Official Kernel Documentation on Cgroups v2
Filesystem as API: A common Linux pattern where you control kernel features by manipulating a virtual filesystem.
OOM Killer: The kernel process that reclaims memory when the system is critically low.

Difficulty: Advanced Time estimate: Weekend Prerequisites: Project 1.

Real world outcome: You will run your C program, which will spawn a child that tries to allocate 100MB of RAM. The program will print a message indicating the child was killed, and dmesg on the host will show a log from the OOM killer.

// Conceptual C code
// (Error handling omitted for brevity)

// 1. Create a cgroup directory
mkdir("/sys/fs/cgroup/my_container", 0755);

// 2. Set the memory limit
int mem_fd = open("/sys/fs/cgroup/my_container/memory.max", O_WRONLY);
write(mem_fd, "50000000", 8); // 50MB
close(mem_fd);

// 3. Get our own PID and add it to the cgroup
pid_t pid = getpid();
int procs_fd = open("/sys/fs/cgroup/my_container/cgroup.procs", O_WRONLY);
// ... write pid to procs_fd ...
close(procs_fd);

// 4. Fork a child process that will be constrained
if (fork() == 0) {
    // This child is now in the cgroup and will be killed if it uses >50MB
    char *mem = malloc(100 * 1024 * 1024); // Allocate 100MB
    memset(mem, 1, 100 * 1024 * 1024); // Use the memory
    printf("I should not have survived!\n");
    exit(0);
}

wait(NULL); // Parent waits and will see the child was killed
printf("Child process terminated as expected.\n");

Learning milestones:

You can create a cgroup and set a limit via file I/O → You understand the cgroupfs API.
Your child process is successfully killed when it exceeds the limit → You have demonstrated kernel-enforced resource control.
You understand the difference between namespaces and cgroups → You have mastered the two core primitives of containerization.

Part 2: Building an IaaS Cloud from Scratch

Project 4: The Hypervisor Controller

File: LEARN_CLOUD_FROM_SCRATCH_IN_C.md
Main Programming Language: C
Alternative Programming Languages: Go, Python
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 1. The “Resume Gold”
Difficulty: Level 3: Advanced
Knowledge Area: Virtualization / Systems Programming
Software or Tool: libvirt C API, KVM/QEMU
Main Book: The official libvirt documentation and examples.

What you’ll build: A C program that uses the libvirt library to connect to the local KVM/QEMU hypervisor, list available virtual machines, start a specific VM by name, and then shut it down.

Why it teaches the fundamentals: This project demystifies how IaaS platforms like OpenStack (specifically, the Nova component) control virtual machines. You’ll learn that there is a standardized API (libvirt) that allows a program to manage the lifecycle of VMs, abstracting away the specifics of the underlying hypervisor (QEMU, Xen, etc.).

Core challenges you’ll face:

Linking against libvirt → maps to gcc myprogram.c -lvirt
Connecting to the hypervisor → maps to virConnectOpen("qemu:///system")
Looking up a VM by name → maps to virDomainLookupByName()
Controlling the VM state → maps to virDomainCreate() (start) and virDomainShutdown()
Handling errors from the library → maps to properly checking for NULL return values and printing libvirt errors

Key Concepts:

Hypervisor: The software that creates and runs virtual machines (e.g., KVM).
libvirt: A standardized, open-source API for managing virtualization platforms. It’s the lingua franca for VM control.
VM Lifecycle: The states of a VM (running, shut off, paused).

Difficulty: Advanced Time estimate: Weekend Prerequisites: Basic C, and have a pre-existing VM configured in QEMU/KVM (you can create one with virt-manager).

Real world outcome: You will have a C program that acts as a remote control for your local virtual machines. Running ./vm-controller start my-ubuntu-vm will cause the VM to boot up.

// Conceptual C code
#include <libvirt/libvirt.h>

int main() {
    virConnectPtr conn = virConnectOpen("qemu:///system");
    if (conn == NULL) { /* handle error */ }

    virDomainPtr domain = virDomainLookupByName(conn, "my-ubuntu-vm");
    if (domain == NULL) { /* handle error */ }

    printf("Starting domain 'my-ubuntu-vm'...
");
    if (virDomainCreate(domain) < 0) {
        /* handle error */
        virDomainFree(domain);
        virConnectClose(conn);
        return 1;
    }

    printf("Domain started. Waiting 10 seconds...
");
    sleep(10);

    printf("Shutting down domain...
");
    virDomainShutdown(domain);

    virDomainFree(domain);
    virConnectClose(conn);
    return 0;
}

Learning milestones:

Your program successfully connects to the hypervisor daemon → You understand the client-server nature of libvirt.
Your program can list, start, and stop a VM → You have replicated the most basic function of an IaaS cloud.
You understand that libvirt is just sending commands to the qemu process → You have peeled back another layer of abstraction.

Project 5: The Distributed Cloud Scheduler

File: LEARN_CLOUD_FROM_SCRATCH_IN_C.md
Main Programming Language: C
Coolness Level: Level 5: Pure Magic (Super Cool)
Business Potential: 4. The “Open Core” Infrastructure
Difficulty: Level 5: Master
Knowledge Area: Distributed Systems / Cloud Architecture
Software or Tool: Sockets API, libvirt, libmicrohttpd
Main Book: “Distributed Systems” by Tanenbaum and van Steen

What you’ll build: A miniature cloud control plane. This is a multi-part project consisting of:

An API Server: An HTTP server (using libmicrohttpd or raw sockets) that listens for requests like POST /vms.
A Scheduler: The API server forwards requests to the scheduler, which decides which node should host the new VM (for now, it can just pick randomly).
A Node Agent (or “compute daemon”): A program that runs on two or more physical/virtual machines. It listens for commands from the scheduler and uses your libvirt code from Project 4 to actually start/stop the VM.

Why it teaches the fundamentals: This project synthesizes everything and builds a micro-version of OpenStack Nova or Kubernetes. You learn how a central API call is translated into a scheduled task that results in a workload (a VM or container) running on a specific machine in a cluster. This is the core loop of all cloud platforms.

Core challenges you’ll face:

Designing a simple RPC protocol → maps to defining the JSON messages between your components (e.g., {"action": "create_vm", "vm_name": "test1"})
Writing a multi-threaded HTTP server → maps to handling concurrent API requests
Implementing a scheduler → maps to maintaining a list of available nodes and choosing one
Building a resilient node agent → maps to a daemon process that can handle commands reliably

Key Concepts:

Control Plane vs. Data Plane: Your API/Scheduler is the control plane. The nodes running the VMs are the data plane.
Scheduler: The “brain” of a distributed system that allocates resources.
Agent-based Architecture: The pattern of having a central controller and distributed agents is universal in cloud systems (e.g., Kubernetes kubelet, OpenStack nova-compute).

Difficulty: Master Time estimate: 1 month+ Prerequisites: All previous projects. Strong knowledge of C, networking (sockets), and threading.

Real world outcome: You will have a complete, end-to-end system. You can run curl -X POST http://localhost:8080/vms on your API server, and a few moments later, see a new virtual machine boot up on one of your registered nodes. You will have built a cloud.

Architecture Diagram:

  curl POST /vms
      │
      ▼
┌──────────────┐   1. Request   ┌───────────┐  2. Command  ┌───────────────┐
│              ├───────────────►│           ├─────────────►│ Node Agent 1  │
│  API Server  │                │ Scheduler │              │ (with libvirt)│
│              │◄───────────────┤           │              └───────────────┘
└──────────────┘   4. Success   └───────────┘  3. Command  ┌───────────────┐
                                              (or fail)    │ Node Agent 2  │
                                                           │ (with libvirt)│
                                                           └───────────────┘

Learning milestones:

Your node agent can receive a command over the network and start a VM → You have a working data plane.
Your API server can receive an HTTP request and forward it to the scheduler → You have a working control plane entry point.
The full end-to-end system works → You can send an API request that results in a VM being created on a scheduled node. You have built a functioning IaaS cloud from scratch and understand exactly how it works at every step.

Summary

Project	Main Component	Key C/Linux Technology	Difficulty
1. The `chroot` Jail	Container Filesystem	`chroot()`, `fork()`	Intermediate
2. Real Isolation with Namespaces	Container Isolation	`clone()`	Advanced
3. Resource Limiting with Cgroups	Container Resource Control	cgroupfs	Advanced
4. The Hypervisor Controller	VM Control	`libvirt`	Advanced
5. The Distributed Cloud Scheduler	Cloud Control Plane	Sockets, `libvirt`	Master

```

Learn Cloud Infrastructure from Scratch in C

Why Learn This Way?

Core Concept Analysis

The Anatomy of a Container (OpenShift/Kubernetes)

The Anatomy of an IaaS Cloud (OpenStack)

Project List: The Path from Process to Cloud

Part 1: Building a Container Runtime from Scratch

Project 1: The chroot Jail

Project 2: Real Isolation with Namespaces

Project 3: Resource Limiting with Cgroups

Part 2: Building an IaaS Cloud from Scratch

Project 4: The Hypervisor Controller

Project 5: The Distributed Cloud Scheduler

Summary

Project 1: The `chroot` Jail