Project 5: File System Server

Build a user-space filesystem server that processes IPC requests for open/read/write.

Quick Reference

Attribute Value
Difficulty Advanced
Time Estimate 2-3 weeks
Language C (Alternatives: Rust)
Prerequisites IPC library, file system basics
Key Topics VFS protocol, inode layout, per-client state

1. Learning Objectives

By completing this project, you will:

  1. Define a filesystem IPC protocol and implement server handlers.
  2. Build basic on-disk structures (inodes, directories, blocks).
  3. Maintain per-client file descriptor state in the server.
  4. Explain how microkernels implement filesystems in user space.

2. Theoretical Foundation

2.1 Core Concepts

  • VFS Layer: Abstracts filesystem operations into a common API.
  • Inodes and Directories: Metadata that maps names to data blocks.
  • Server State: File offsets, open flags, and per-client tables.
  • IPC-Based Syscalls: Every file operation becomes a message.

2.2 Why This Matters

In microkernels, the filesystem is just another service. If it crashes, the kernel stays alive and can restart the server.

2.3 Historical Context / Background

QNX and MINIX route POSIX file operations through user-space servers. FUSE popularized user-space filesystems on Linux for similar reasons.

2.4 Common Misconceptions

  • “Filesystems must run in kernel mode.” They can run in user space with IPC.
  • “IPC makes it unusably slow.” With good design and batching, it’s acceptable.

3. Project Specification

3.1 What You Will Build

A filesystem server that handles open/read/write/close via IPC. It implements a simple on-disk layout (FAT-like or ext2-like) and exposes a mount path to clients.

3.2 Functional Requirements

  1. IPC protocol for open/read/write/close/stat.
  2. Block storage layer for reading/writing blocks.
  3. Directory support for file lookup.
  4. Per-client FD tables for open files.
  5. Basic concurrency for multiple clients.

3.3 Non-Functional Requirements

  • Consistency: Writes update metadata safely.
  • Reliability: Server handles malformed requests gracefully.
  • Usability: POSIX-like semantics for clients.

3.4 Example Usage / Output

int fd = fs_open("/hello.txt", O_RDONLY);
char buf[64];
fs_read(fd, buf, sizeof(buf));
fs_close(fd);

3.5 Real World Outcome

$ ./fs_server disk.img &
[fs] mounted microfs
[fs] listening on endpoint "vfs"

$ ./fs_client cat /hello.txt
[fs] OPEN /hello.txt
[fs] READ fd=3, 64 bytes
Hello, microkernel world!

4. Solution Architecture

4.1 High-Level Design

┌──────────────┐  IPC   ┌──────────────┐  blocks  ┌──────────────┐
│   Client     │ ─────▶ │  FS Server   │ ───────▶ │ Block Store  │
└──────────────┘ ◀───── │  (VFS ops)   │ ◀─────── │ (disk.img)   │
                         └──────────────┘         └──────────────┘

4.2 Key Components

Component Responsibility Key Decisions
IPC protocol Define request/response Fixed structs vs variable
VFS layer Dispatch operations Table of handlers
Block store Read/write blocks Cache vs direct
Metadata Inodes/directories Simple or ext2-like

4.3 Data Structures

typedef struct {
    uint32_t inode;
    uint32_t size;
    uint32_t blocks[12];
} inode_t;

typedef struct {
    uint32_t inode;
    uint32_t offset;
    uint32_t flags;
} open_file_t;

4.4 Algorithm Overview

Key Algorithm: open(path)

  1. Parse path and traverse directory entries.
  2. Load inode and validate permissions.
  3. Allocate an FD entry for client.

Complexity Analysis:

  • Time: O(depth) for path traversal
  • Space: O(N) for open file tables

5. Implementation Guide

5.1 Development Environment Setup

cc -O2 -g -o fs_server *.c

5.2 Project Structure

fs_server/
├── src/
│   ├── ipc.c
│   ├── vfs.c
│   ├── inode.c
│   ├── block.c
│   └── main.c
├── include/
│   └── fs.h
└── tests/
    └── test_fs.c

5.3 The Core Question You’re Answering

“How can a filesystem be just another user process without losing POSIX semantics?”

5.4 Concepts You Must Understand First

Stop and research these before coding:

  1. Inode and directory structures
  2. File descriptor semantics
  3. Path traversal
  4. Caching vs consistency

5.5 Questions to Guide Your Design

  1. How will you represent directory entries on disk?
  2. How will you handle concurrent reads/writes?
  3. Will you implement caching? Where?
  4. How will you report errors to clients?

5.6 Thinking Exercise

Design a Minimal On-Disk Layout

Sketch a disk image layout: superblock, inode table, data blocks. How do you find a file named /hello.txt?

5.7 The Interview Questions They’ll Ask

  1. “What is the role of the VFS layer?”
  2. “How does a filesystem server manage per-client state?”
  3. “What is the difference between inode metadata and data blocks?”

5.8 Hints in Layers

Hint 1: Start with a RAM-backed FS Prototype with an in-memory block device.

Hint 2: Add a simple on-disk format Write the superblock and inode table to a file.

Hint 3: Add directories last Implement single-file support before full paths.

5.9 Books That Will Help

Topic Book Chapter
File system design OSTEP Ch. 39-42
VFS concepts Linux VFS docs Overview

5.10 Implementation Phases

Phase 1: Foundation (5 days)

Goals:

  • IPC protocol and block store

Tasks:

  1. Define request/response structs.
  2. Implement block read/write against disk image.

Checkpoint: Raw block reads work.

Phase 2: Core Functionality (7 days)

Goals:

  • Inodes and file read/write

Tasks:

  1. Implement inode lookup by index.
  2. Support open/read/write/close.

Checkpoint: cat reads a file via IPC.

Phase 3: Polish & Edge Cases (5 days)

Goals:

  • Directories and concurrency

Tasks:

  1. Implement directory entries and path traversal.
  2. Add per-client FD tables.

Checkpoint: Multiple clients can read different files.

5.11 Key Implementation Decisions

Decision Options Recommendation Rationale
Storage backend File, RAM File-backed image More realistic
Directory format Fixed, variable Fixed entries Simpler parsing
Cache None, LRU Small LRU Performance boost

6. Testing Strategy

6.1 Test Categories

Category Purpose Examples
Unit Tests Block read/write read block 0
Integration Tests IPC file ops open/read/close
Concurrency Tests Multiple clients two clients read

6.2 Critical Test Cases

  1. Open non-existent file returns error.
  2. Read past EOF returns 0 bytes.
  3. Concurrent reads do not corrupt offsets.

6.3 Test Data

Files: /hello.txt, /dir/a.txt
Sizes: 1B, 4KB, 1MB

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

Pitfall Symptom Solution
Incorrect inode math Wrong file contents Verify block index math
FD reuse bugs Wrong offsets Store offset per FD
Path parsing errors ENOENT Normalize path segments

7.2 Debugging Strategies

  • Add a fsck-like validator for the image.
  • Log IPC ops with file and inode numbers.

7.3 Performance Traps

Reading single bytes via IPC is expensive. Implement buffered reads on client side.

8. Extensions & Challenges

8.1 Beginner Extensions

  • Add stat and readdir.
  • Add a simple mkfs tool.

8.2 Intermediate Extensions

  • Add write-ahead logging.
  • Add caching with invalidation.

8.3 Advanced Extensions

  • Snapshotting and rollback.
  • Fault recovery via Project 13 supervisor.

9. Real-World Connections

9.1 Industry Applications

  • QNX: Resource managers for file IO.
  • Fuchsia: Filesystems as components.
  • FUSE: https://github.com/libfuse/libfuse
  • MINIX VFS: https://www.minix3.org/

9.3 Interview Relevance

Filesystem questions commonly test inode and VFS understanding.

10. Resources

10.1 Essential Reading

  • OSTEP - File systems chapters.
  • FUSE Documentation - IPC filesystem design.

10.2 Video Resources

  • OS course lectures on filesystem internals.

10.3 Tools & Documentation

  • strace for observing file calls.
  • hexdump for inspecting disk images.
  • Project 1: IPC transport.
  • Project 6: Network stack as a service.

11. Self-Assessment Checklist

11.1 Understanding

  • I can explain inode vs data blocks.
  • I can describe how IPC replaces syscalls.

11.2 Implementation

  • open/read/write/close work correctly.
  • Multiple clients maintain independent offsets.

11.3 Growth

  • I can outline how to add journaling.

12. Submission / Completion Criteria

Minimum Viable Completion:

  • IPC protocol implemented.
  • Single-file open/read/close works.

Full Completion:

  • Directory traversal works.
  • Multiple clients supported.

Excellence (Going Above & Beyond):

  • Journaling or crash recovery.
  • Performance metrics documented.

This guide was generated from LEARN_MICROKERNELS.md. For the complete learning path, see the parent directory.