Project 5: Alpine Diskless Kiosk System

Build a read-only, diskless Alpine Linux system that runs entirely from RAM - perfect for kiosks, digital signage, firewalls, or embedded IoT devices. Your system boots from USB/SD, loads into RAM, and runs tamper-proof with configuration persistence via LBU.

Quick Reference

Attribute Value
Difficulty ★★★☆☆ Advanced
Time Estimate 2 weeks (40-60 hours)
Language Shell (sh/ash)
Prerequisites Linux administration, networking basics, shell scripting
Key Topics Diskless mode, LBU (Local Backup Utility), apkovl, tmpfs, RAM-based boot, OpenRC, Raspberry Pi

1. Learning Objectives

After completing this project, you will be able to:

  • Explain why Alpine’s diskless mode is unique and when to use it
  • Configure Alpine to boot entirely from RAM with the root filesystem on tmpfs
  • Use LBU (Local Backup Utility) to persist configuration changes across reboots
  • Understand and create apkovl (Alpine Package Overlay) files
  • Set up APK package caching for offline operation and faster boots
  • Configure a read-only boot medium to prevent wear on SD cards and USB drives
  • Build production-ready systems for kiosks, firewalls, and embedded devices
  • Debug boot issues and recover from misconfigured diskless systems
  • Adapt the system for Raspberry Pi and other ARM-based devices

2. Theoretical Foundation

2.1 Core Concepts

What is Diskless Mode?

Diskless mode is Alpine’s unique installation mode where the entire operating system runs from RAM (tmpfs). Unlike traditional Linux installations that mount the root filesystem from disk, diskless Alpine:

TRADITIONAL LINUX BOOT                    ALPINE DISKLESS BOOT
┌─────────────────────────┐              ┌─────────────────────────┐
│                         │              │                         │
│  ┌──────────────────┐   │              │  ┌──────────────────┐   │
│  │   Hard Drive     │   │              │  │   USB/SD Card    │   │
│  │   ───────────    │   │              │  │   (read-only)    │   │
│  │   /bin           │   │              │  │                  │   │
│  │   /etc           │   │              │  │   alpine.iso     │   │
│  │   /usr    ◄──────┼───┼── mounted   │  │   apkovl.tar.gz  │   │
│  │   /var           │   │   directly   │  │   apk cache      │   │
│  │   ...            │   │              │  └────────┬─────────┘   │
│  └──────────────────┘   │              │           │ load at     │
│                         │              │           │ boot        │
│  Every write goes       │              │           ▼             │
│  to physical disk       │              │  ┌──────────────────┐   │
│                         │              │  │      RAM         │   │
└─────────────────────────┘              │  │    (tmpfs)       │   │
                                         │  │                  │   │
                                         │  │   /bin           │   │
                                         │  │   /etc ◄─── all  │   │
                                         │  │   /usr     writes│   │
                                         │  │   /var     go    │   │
                                         │  │   ...      here  │   │
                                         │  └──────────────────┘   │
                                         │                         │
                                         │  Boot medium never      │
                                         │  written after boot     │
                                         └─────────────────────────┘

Key Characteristics:

  • Boot medium (USB/SD) is mounted read-only after initial load
  • All system files copied to RAM (tmpfs)
  • Changes are lost on reboot unless explicitly saved
  • No disk wear during operation
  • Instant “factory reset” by rebooting

The Alpine Boot Process (Diskless)

Understanding the boot sequence is essential for debugging and customization:

┌─────────────────────────────────────────────────────────────────────┐
│                     ALPINE DISKLESS BOOT SEQUENCE                    │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  1. BIOS/UEFI Loads Bootloader                                       │
│     │                                                                │
│     ▼                                                                │
│  2. Bootloader (syslinux/grub) Loads Kernel + initramfs             │
│     │   - Kernel: /boot/vmlinuz-lts                                  │
│     │   - Initramfs: /boot/initramfs-lts                             │
│     │   - Boot params: alpine_dev=LABEL=ALPINE                       │
│     │                                                                │
│     ▼                                                                │
│  3. initramfs Runs /init Script                                      │
│     │   - Finds boot medium (alpine_dev)                             │
│     │   - Mounts medium at /media/<device>                           │
│     │   - Creates tmpfs at /                                         │
│     │                                                                │
│     ▼                                                                │
│  4. Load apkovl (Alpine Overlay)                                     │
│     │   - Searches for <hostname>.apkovl.tar.gz                      │
│     │   - Extracts to / (overlays /etc, /root, etc.)                 │
│     │   - Applies saved configuration                                │
│     │                                                                │
│     ▼                                                                │
│  5. Install Packages from Cache                                      │
│     │   - Reads /etc/apk/world                                       │
│     │   - Installs packages from local cache                         │
│     │   - No network needed if cache is complete                     │
│     │                                                                │
│     ▼                                                                │
│  6. Switch Root to tmpfs                                             │
│     │   - pivot_root to new filesystem                               │
│     │   - Execute /sbin/init (OpenRC)                                │
│     │                                                                │
│     ▼                                                                │
│  7. OpenRC Starts Services                                           │
│     │   - Runs /etc/runlevels/boot/*                                 │
│     │   - Runs /etc/runlevels/default/*                              │
│     │   - System ready!                                              │
│     │                                                                │
│     ▼                                                                │
│  8. SYSTEM RUNNING FROM RAM                                          │
│     - Boot medium: read-only at /media/usb or /media/mmcblk0p1       │
│     - All changes in RAM (lost on reboot)                            │
│     - Use `lbu commit` to save changes                               │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

LBU (Local Backup Utility)

LBU is Alpine’s configuration persistence mechanism. It creates compressed tarballs of your configuration:

LBU WORKFLOW
┌──────────────────────────────────────────────────────────────────────┐
│                                                                       │
│  1. You make changes to the system                                    │
│     └─► Edit /etc/network/interfaces                                  │
│     └─► Add user with adduser                                         │
│     └─► Install packages with apk                                     │
│                                                                       │
│  2. Run `lbu status` to see what changed                              │
│     ┌─────────────────────────────────────────────────────────────┐  │
│     │ A etc/network/interfaces                                     │  │
│     │ A etc/passwd                                                 │  │
│     │ A etc/shadow                                                 │  │
│     │ A etc/apk/world                                              │  │
│     └─────────────────────────────────────────────────────────────┘  │
│                                                                       │
│  3. Run `lbu commit` to save                                          │
│     │                                                                 │
│     ▼                                                                 │
│  4. LBU creates: <hostname>.apkovl.tar.gz                             │
│     │   Contains: /etc/* (modified files)                             │
│     │             /root/*                                             │
│     │             Custom paths (lbu include)                          │
│     │                                                                 │
│     ▼                                                                 │
│  5. Saved to boot medium: /media/usb/<hostname>.apkovl.tar.gz         │
│                                                                       │
│  6. On next boot: initramfs extracts this overlay                     │
│     └─► Your configuration is restored!                               │
│                                                                       │
└──────────────────────────────────────────────────────────────────────┘

Key LBU Commands:

Command Description
lbu status Show files that will be saved (A=added, M=modified, D=deleted)
lbu list List files currently in the overlay
lbu commit Save changes to boot medium
lbu commit -d Save and show what was saved
lbu include /path Add custom path to backup
lbu exclude /path Remove path from backup
lbu pkg Create a standalone apkovl tarball

apkovl (Alpine Package Overlay)

The apkovl is a gzipped tarball containing your system’s configuration state:

APKOVL STRUCTURE
┌─────────────────────────────────────────────────────────────────────┐
│                                                                      │
│  mykiosk.apkovl.tar.gz                                               │
│  ├── etc/                                                            │
│  │   ├── apk/                                                        │
│  │   │   ├── world             ◄── List of installed packages       │
│  │   │   └── repositories      ◄── APK repository URLs              │
│  │   ├── conf.d/                                                     │
│  │   │   └── local             ◄── LBU configuration                │
│  │   ├── hostname              ◄── System hostname                  │
│  │   ├── hosts                 ◄── Local DNS entries                │
│  │   ├── network/                                                    │
│  │   │   └── interfaces        ◄── Network configuration            │
│  │   ├── passwd                ◄── User accounts                    │
│  │   ├── shadow                ◄── Password hashes                  │
│  │   ├── group                 ◄── Group definitions                │
│  │   ├── ssh/                                                        │
│  │   │   ├── sshd_config       ◄── SSH server configuration         │
│  │   │   └── ssh_host_*        ◄── SSH host keys                    │
│  │   ├── local.d/                                                    │
│  │   │   └── kiosk.start       ◄── Custom startup script            │
│  │   └── runlevels/                                                  │
│  │       └── default/          ◄── Enabled services (symlinks)      │
│  └── root/                                                           │
│      └── .ssh/                                                       │
│          └── authorized_keys   ◄── SSH public keys for root         │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

How apkovl is Found at Boot:

The initramfs searches for the overlay in this order:

  1. <hostname>.apkovl.tar.gz on boot medium
  2. localhost.apkovl.tar.gz if hostname not set
  3. URL specified in apkovl= boot parameter

APK Cache

Without a package cache, a diskless system needs network access at every boot to install packages. The cache stores downloaded packages locally:

APK CACHE SETUP
┌─────────────────────────────────────────────────────────────────────┐
│                                                                      │
│  Boot Medium (/media/usb)                                            │
│  ├── boot/                                                           │
│  │   ├── vmlinuz-lts                                                 │
│  │   └── initramfs-lts                                               │
│  ├── apks/                     ◄── APK CACHE DIRECTORY               │
│  │   ├── APKINDEX.*.tar.gz     ◄── Package index                    │
│  │   ├── alpine-base-*.apk     ◄── Cached packages                  │
│  │   ├── busybox-*.apk                                               │
│  │   ├── openrc-*.apk                                                │
│  │   └── ...                                                         │
│  └── mykiosk.apkovl.tar.gz                                           │
│                                                                      │
│  Configuration: /etc/apk/cache → /media/usb/apks                     │
│                                                                      │
│  BOOT WITH CACHE:                                                    │
│  ┌─────────────────────────────────────────────────────────────┐    │
│  │  1. Read /etc/apk/world (from apkovl)                        │    │
│  │  2. For each package:                                        │    │
│  │     - Check /media/usb/apks/ for cached .apk                 │    │
│  │     - If found: install from cache (NO NETWORK)              │    │
│  │     - If not: download and install (NEEDS NETWORK)           │    │
│  │  3. System boots faster, works offline                       │    │
│  └─────────────────────────────────────────────────────────────┘    │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

2.2 Why This Matters

Use Cases for Diskless Alpine

  1. Kiosks and Digital Signage
    • Read-only root prevents tampering
    • Reboot returns to known state
    • No disk wear from continuous operation
    • Easy mass deployment (clone USB/SD)
  2. Firewalls and Network Appliances
    • Security: no persistent malware
    • RAM-based for speed
    • Instant recovery: reboot
    • Predictable state
  3. Embedded Systems and IoT
    • SD card longevity (no writes)
    • Small footprint
    • Fast boot times
    • Reliable operation
  4. Development and Testing
    • Ephemeral environments
    • Reproducible setups
    • Quick iteration
  5. Raspberry Pi Applications
    • SD cards fail from write wear
    • Diskless extends SD life dramatically
    • Power loss won’t corrupt filesystem

SD Card Wear Prevention

SD cards have limited write cycles. Traditional Linux systems write constantly:

WRITE SOURCES IN TRADITIONAL LINUX
┌─────────────────────────────────────────────────────────────────────┐
│                                                                      │
│  /var/log/*        ─► Constant writes (syslog, app logs)            │
│  /var/tmp/*        ─► Temporary files                               │
│  /tmp/*            ─► More temporary files                          │
│  /var/lib/apt/*    ─► Package manager state                         │
│  Journald          ─► Binary journal writes                         │
│  SQLite DBs        ─► Many apps use SQLite                          │
│  Browser caches    ─► Constant updates                              │
│  Swap              ─► Virtual memory                                │
│                                                                      │
│  Result: SD card fails in months                                    │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

DISKLESS MODE
┌─────────────────────────────────────────────────────────────────────┐
│                                                                      │
│  All writes → RAM (tmpfs)                                           │
│  SD card → Read-only after boot                                     │
│  Result: SD card lasts years                                        │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

2.3 Historical Context

Alpine Linux was created in 2005 by Natanael Copa, originally as a fork of LEAF (Linux Embedded Appliance Firewall). The diskless mode was a core design goal from the beginning, targeting:

  • Router/Firewall appliances: Boot from USB, run from RAM
  • Embedded systems: Where flash storage has limited write cycles
  • Live systems: Portable, reproducible environments

The design philosophy was influenced by:

  • OpenWrt: Embedded Linux for routers
  • LEAF/LRP: Early embedded Linux distributions
  • Unix philosophy: Simple, modular, text-based configuration

Today, Alpine’s diskless mode is used in:

  • Edge computing devices
  • Network appliances (pfSense inspiration)
  • Containerized environments (Docker base images)
  • Raspberry Pi projects

2.4 Common Misconceptions

Misconception 1: “Diskless means no persistent storage”

  • Reality: LBU provides persistence for configuration. Only the runtime state is ephemeral. You choose what to save.

Misconception 2: “RAM usage is huge because the whole OS is in memory”

  • Reality: Alpine base is ~5 MB. Even with packages, most systems use <100 MB. Unused RAM becomes file cache.

Misconception 3: “Diskless is complicated to set up”

  • Reality: Alpine’s setup-alpine can configure diskless mode directly. The complexity is in understanding, not execution.

Misconception 4: “You can’t update packages in diskless mode”

  • Reality: You can install packages at runtime. They live in RAM until reboot. Run lbu commit after apk add to persist the change (it saves /etc/apk/world with the new package list).

Misconception 5: “Diskless mode requires special hardware”

  • Reality: Any bootable device works: USB flash drive, SD card, CD-ROM, network boot (PXE). Standard hardware is fine.

Misconception 6: “Changes are automatically saved”

  • Reality: You must explicitly run lbu commit. This is a feature (predictable state), not a bug.

3. Project Specification

3.1 What You Will Build

A complete diskless kiosk system with these components:

  1. Bootable Alpine USB/SD image that loads into RAM
  2. Custom configuration overlay with networking, users, and services
  3. Kiosk application (web browser in kiosk mode, or custom app)
  4. Automatic updates with persistence
  5. Remote management via SSH
  6. Monitoring and logging (in RAM with optional persistence)

System Architecture:

┌─────────────────────────────────────────────────────────────────────┐
│                     DISKLESS KIOSK ARCHITECTURE                      │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  BOOT MEDIUM (USB/SD - Read Only After Boot)                         │
│  ┌────────────────────────────────────────────────────────────────┐ │
│  │                                                                 │ │
│  │  /boot/                          /apks/                         │ │
│  │  ├── vmlinuz-lts                 ├── alpine-base-*.apk         │ │
│  │  ├── initramfs-lts               ├── chromium-*.apk            │ │
│  │  └── syslinux/                   ├── xorg-*.apk                │ │
│  │                                   └── ...                        │ │
│  │                                                                 │ │
│  │  /kiosk.apkovl.tar.gz                                           │ │
│  │  (Configuration overlay)                                        │ │
│  │                                                                 │ │
│  └────────────────────────────────────────────────────────────────┘ │
│                                                                      │
│  RUNTIME (RAM - tmpfs)                                               │
│  ┌────────────────────────────────────────────────────────────────┐ │
│  │                                                                 │ │
│  │  /etc/                           /var/                          │ │
│  │  ├── network/interfaces          ├── log/ (in RAM)             │ │
│  │  ├── local.d/kiosk.start         └── tmp/                       │ │
│  │  ├── apk/world                                                  │ │
│  │  └── hostname                   /home/kiosk/                    │ │
│  │                                   └── (browser profile)         │ │
│  │                                                                 │ │
│  │  SERVICES RUNNING:                                              │ │
│  │  ├── sshd (remote management)                                   │ │
│  │  ├── dbus (for X11/Wayland)                                     │ │
│  │  ├── xorg/cage (display server)                                 │ │
│  │  └── chromium (kiosk browser)                                   │ │
│  │                                                                 │ │
│  └────────────────────────────────────────────────────────────────┘ │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

3.2 Functional Requirements

Core System:

  • Boot from USB or SD card into RAM
  • Root filesystem runs entirely on tmpfs
  • Boot medium is read-only after initial load
  • System boots without network connectivity
  • Configuration persists across reboots via LBU

Networking:

  • DHCP for primary network interface
  • Optional static IP configuration
  • DNS resolution works correctly
  • SSH access for remote management

Kiosk Application:

  • Auto-login to kiosk user (no password prompt)
  • Auto-start display server on boot
  • Auto-launch kiosk application (browser/app)
  • Kiosk runs fullscreen, no window decorations
  • No access to underlying OS from kiosk interface

Management:

  • SSH access with key-based authentication
  • lbu commit saves configuration changes
  • Package cache populated for offline operation
  • Update procedure documented

3.3 Non-Functional Requirements

  • Boot time under 30 seconds (goal: under 15 seconds)
  • RAM usage under 256 MB at idle (512 MB for graphical kiosk)
  • Boot medium capacity under 1 GB
  • Support x86_64 and ARM (Raspberry Pi)
  • No network required for normal operation
  • Survive power loss without corruption

3.4 Example Usage / Output

Building the Image:

# Download Alpine extended ISO
$ wget https://dl-cdn.alpinelinux.org/alpine/v3.19/releases/x86_64/alpine-extended-3.19.0-x86_64.iso

# Write to USB
$ sudo dd if=alpine-extended-3.19.0-x86_64.iso of=/dev/sdb bs=4M status=progress

# Boot from USB and run setup
# Select 'diskless' mode during setup-alpine

After Boot - Checking Diskless Mode:

# Verify root is tmpfs
$ mount | grep "on / "
tmpfs on / type tmpfs (rw,relatime)

# Verify boot medium is mounted
$ mount | grep "/media"
/dev/sdb1 on /media/usb type vfat (ro,relatime,fmask=0022,dmask=0022)

# Check LBU configuration
$ cat /etc/lbu/lbu.conf
LBU_MEDIA=usb

# See what will be saved
$ lbu status
A etc/network/interfaces
A etc/hostname
A etc/apk/world
...

# Save configuration
$ lbu commit -d
Creating backup of mykiosk.apkovl.tar.gz...
Committing config to media...
  etc/apk/world
  etc/hostname
  etc/network/interfaces
  ...
Done.

Runtime State:

# Memory usage
$ free -h
              total        used        free      shared  buff/cache   available
Mem:           1.9G        128M        1.5G         12M        268M        1.7G

# Disk usage (RAM-based)
$ df -h /
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           1.0G  120M  904M  12% /

# Boot medium usage
$ df -h /media/usb
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb1       950M  180M  770M  19% /media/usb

Kiosk in Action:

┌─────────────────────────────────────────────────────────────────────┐
│ Chrome - http://dashboard.local                                      │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│                    ┌─────────────────────────────┐                   │
│                    │                             │                   │
│                    │      YOUR DASHBOARD         │                   │
│                    │       OR SIGNAGE            │                   │
│                    │                             │                   │
│                    │     (No window decorations) │                   │
│                    │     (No taskbar visible)    │                   │
│                    │     (Fullscreen only)       │                   │
│                    │                             │                   │
│                    └─────────────────────────────┘                   │
│                                                                      │
│                  No access to desktop or terminal                    │
│                  Reboot returns to this state                        │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

3.5 Real World Outcome

When you complete this project:

  • You will have a production-ready kiosk system
  • You can deploy dozens of identical kiosks by cloning the boot medium
  • The system survives power loss and always boots to a known state
  • You understand Alpine’s unique diskless architecture deeply
  • You can adapt this for firewalls, IoT devices, or any embedded use case
  • You have practical experience with RAM-based Linux systems

4. Solution Architecture

4.1 High-Level Design

┌─────────────────────────────────────────────────────────────────────┐
│                    DISKLESS KIOSK SYSTEM DESIGN                      │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  LAYER 1: BOOT INFRASTRUCTURE                                        │
│  ┌────────────────────────────────────────────────────────────────┐ │
│  │                                                                 │ │
│  │  ┌─────────────┐   ┌─────────────┐   ┌─────────────┐           │ │
│  │  │   BIOS/     │   │  Syslinux/  │   │   Kernel    │           │ │
│  │  │   UEFI      │──▶│   GRUB      │──▶│ + initramfs │           │ │
│  │  └─────────────┘   └─────────────┘   └──────┬──────┘           │ │
│  │                                              │                  │ │
│  └──────────────────────────────────────────────┼──────────────────┘ │
│                                                 │                    │
│  LAYER 2: INITRAMFS (Early Boot)               ▼                    │
│  ┌────────────────────────────────────────────────────────────────┐ │
│  │                                                                 │ │
│  │  ┌─────────────┐   ┌─────────────┐   ┌─────────────┐           │ │
│  │  │ Find boot   │   │  Create     │   │  Extract    │           │ │
│  │  │  medium     │──▶│   tmpfs     │──▶│   apkovl    │           │ │
│  │  └─────────────┘   └─────────────┘   └──────┬──────┘           │ │
│  │                                              │                  │ │
│  └──────────────────────────────────────────────┼──────────────────┘ │
│                                                 │                    │
│  LAYER 3: PACKAGE INSTALLATION                 ▼                    │
│  ┌────────────────────────────────────────────────────────────────┐ │
│  │                                                                 │ │
│  │  ┌─────────────┐   ┌─────────────┐   ┌─────────────┐           │ │
│  │  │ Read world  │   │  Install    │   │  Configure  │           │ │
│  │  │   file      │──▶│  packages   │──▶│  services   │           │ │
│  │  └─────────────┘   │ from cache  │   └──────┬──────┘           │ │
│  │                    └─────────────┘          │                  │ │
│  └─────────────────────────────────────────────┼──────────────────┘ │
│                                                │                    │
│  LAYER 4: OPENRC INIT                          ▼                    │
│  ┌────────────────────────────────────────────────────────────────┐ │
│  │                                                                 │ │
│  │  ┌─────────────┐   ┌─────────────┐   ┌─────────────┐           │ │
│  │  │  sysinit    │   │    boot     │   │   default   │           │ │
│  │  │  runlevel   │──▶│  runlevel   │──▶│  runlevel   │           │ │
│  │  └─────────────┘   └─────────────┘   └──────┬──────┘           │ │
│  │                                              │                  │ │
│  └──────────────────────────────────────────────┼──────────────────┘ │
│                                                 │                    │
│  LAYER 5: KIOSK APPLICATION                    ▼                    │
│  ┌────────────────────────────────────────────────────────────────┐ │
│  │                                                                 │ │
│  │  ┌─────────────┐   ┌─────────────┐   ┌─────────────┐           │ │
│  │  │  Auto-login │   │   Display   │   │    Kiosk    │           │ │
│  │  │   (agetty)  │──▶│   Server    │──▶│    App      │           │ │
│  │  └─────────────┘   │ (X11/Cage)  │   │ (Chromium)  │           │ │
│  │                    └─────────────┘   └─────────────┘           │ │
│  │                                                                 │ │
│  └────────────────────────────────────────────────────────────────┘ │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

4.2 Key Components

1. Boot Medium Layout

USB/SD CARD FILESYSTEM (FAT32 or ext4)
/
├── boot/
│   ├── syslinux/
│   │   ├── syslinux.cfg          # Bootloader config
│   │   ├── ldlinux.sys           # Syslinux bootloader
│   │   └── libutil.c32           # Syslinux modules
│   ├── vmlinuz-lts               # Linux kernel
│   ├── initramfs-lts             # Initial ramdisk
│   └── modloop-lts               # Kernel modules
├── apks/                          # Package cache
│   ├── APKINDEX.tar.gz
│   └── *.apk                      # Cached packages
└── kiosk.apkovl.tar.gz           # Configuration overlay

2. tmpfs Root Structure (Runtime)

/ (tmpfs - RAM)
├── bin/         -> busybox symlinks
├── etc/
│   ├── apk/
│   │   ├── world                  # Package list
│   │   ├── repositories           # Repo URLs
│   │   └── cache -> /media/usb/apks
│   ├── local.d/
│   │   └── kiosk.start            # Kiosk startup script
│   ├── lbu/
│   │   └── lbu.conf               # LBU configuration
│   └── network/
│       └── interfaces             # Network config
├── home/
│   └── kiosk/                     # Kiosk user home
├── media/
│   └── usb/                       # Boot medium (read-only)
├── run/                           # Runtime state
├── tmp/                           # Temporary files
└── var/
    ├── cache/apk/ -> /media/usb/apks
    └── log/                       # Logs (in RAM)

3. Kiosk User Environment

/home/kiosk/
├── .profile                       # Shell profile
├── .xinitrc                       # X11 startup script
├── .config/
│   └── chromium/                  # Browser profile
└── .cache/                        # Browser cache (RAM)

4.3 Data Flow

BOOT DATA FLOW
┌─────────────────────────────────────────────────────────────────────┐
│                                                                      │
│  1. BIOS reads bootloader from boot medium                          │
│     /boot/syslinux/ldlinux.sys                                       │
│           │                                                          │
│           ▼                                                          │
│  2. Syslinux loads kernel and initramfs                             │
│     /boot/vmlinuz-lts + /boot/initramfs-lts                          │
│           │                                                          │
│           ▼                                                          │
│  3. Initramfs creates tmpfs, mounts boot medium                     │
│     tmpfs → /                                                        │
│     /dev/sdb1 → /media/usb (read-only)                               │
│           │                                                          │
│           ▼                                                          │
│  4. Initramfs extracts apkovl to tmpfs                              │
│     /media/usb/kiosk.apkovl.tar.gz → /etc/*, /root/*                 │
│           │                                                          │
│           ▼                                                          │
│  5. Initramfs installs packages from cache                          │
│     /etc/apk/world + /media/usb/apks/*.apk → /usr/bin/*              │
│           │                                                          │
│           ▼                                                          │
│  6. OpenRC starts services from /etc/runlevels/                      │
│     networking, sshd, local scripts                                  │
│           │                                                          │
│           ▼                                                          │
│  7. local.d/kiosk.start launches kiosk                              │
│     agetty → login → .profile → .xinitrc → chromium                  │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

4.4 Configuration Persistence Flow

LBU COMMIT DATA FLOW
┌─────────────────────────────────────────────────────────────────────┐
│                                                                      │
│  RUNTIME STATE (tmpfs at /)                                          │
│  ┌────────────────────────────────────────────────────────────────┐ │
│  │                                                                 │ │
│  │  /etc/network/interfaces    (modified)                          │ │
│  │  /etc/apk/world             (modified after apk add)            │ │
│  │  /etc/hostname              (modified)                          │ │
│  │  /root/.ssh/authorized_keys (added)                             │ │
│  │                                                                 │ │
│  └───────────────────────────────┬────────────────────────────────┘ │
│                                  │                                   │
│                                  │ lbu commit                        │
│                                  ▼                                   │
│  BACKUP ON BOOT MEDIUM (/media/usb)                                  │
│  ┌────────────────────────────────────────────────────────────────┐ │
│  │                                                                 │ │
│  │  kiosk.apkovl.tar.gz.bak    (previous backup)                   │ │
│  │  kiosk.apkovl.tar.gz        (new backup)                        │ │
│  │                                                                 │ │
│  │  Contents:                                                      │ │
│  │  ├── etc/network/interfaces                                     │ │
│  │  ├── etc/apk/world                                              │ │
│  │  ├── etc/hostname                                               │ │
│  │  └── root/.ssh/authorized_keys                                  │ │
│  │                                                                 │ │
│  └────────────────────────────────────────────────────────────────┘ │
│                                                                      │
│  ON NEXT BOOT:                                                       │
│  ┌────────────────────────────────────────────────────────────────┐ │
│  │                                                                 │ │
│  │  Initramfs extracts kiosk.apkovl.tar.gz to /                    │ │
│  │  → Configuration is restored automatically                      │ │
│  │                                                                 │ │
│  └────────────────────────────────────────────────────────────────┘ │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

5. Implementation Guide

5.1 Development Environment Setup

Required Tools:

For building and testing on your main machine:

# macOS
brew install qemu cdrtools wget

# Ubuntu/Debian
sudo apt update
sudo apt install qemu-system-x86 qemu-utils wget genisoimage

# Fedora/RHEL
sudo dnf install qemu-system-x86 qemu-img wget genisoimage

Download Alpine:

# Create project directory
mkdir -p ~/alpine-kiosk && cd ~/alpine-kiosk

# Download Alpine Extended (includes more packages for diskless)
# For x86_64:
wget https://dl-cdn.alpinelinux.org/alpine/v3.19/releases/x86_64/alpine-extended-3.19.0-x86_64.iso

# For Raspberry Pi (aarch64):
wget https://dl-cdn.alpinelinux.org/alpine/v3.19/releases/aarch64/alpine-rpi-3.19.0-aarch64.tar.gz

QEMU Test Environment:

# Create a virtual USB disk for testing
qemu-img create -f raw usb.img 1G

# Boot Alpine ISO with the virtual USB attached
qemu-system-x86_64 \
    -m 1024 \
    -cdrom alpine-extended-3.19.0-x86_64.iso \
    -drive file=usb.img,format=raw,if=virtio \
    -boot d \
    -enable-kvm \
    -display gtk

5.2 Project Structure

alpine-kiosk/
├── docs/                          # Your documentation
│   └── SETUP.md
├── overlay/                       # Custom files to include
│   ├── etc/
│   │   ├── local.d/
│   │   │   └── kiosk.start        # Kiosk startup script
│   │   ├── network/
│   │   │   └── interfaces         # Network configuration
│   │   └── hostname               # System hostname
│   └── home/
│       └── kiosk/
│           └── .xinitrc           # X11 startup
├── scripts/                       # Helper scripts
│   ├── build-image.sh             # Automation script
│   └── test-qemu.sh               # QEMU testing
├── alpine-extended-*.iso          # Downloaded ISO
└── usb.img                        # Test USB image

5.3 The Core Question You’re Answering

“How do you create a Linux system that runs entirely from RAM, persists configuration selectively, and survives power loss without corruption?”

This project answers:

  • Why would you want a RAM-based system?
  • How does Alpine’s boot process differ from traditional Linux?
  • What is the mechanism for loading configuration at boot?
  • How do you balance ephemerality with persistence?
  • What trade-offs exist between RAM usage and features?

5.4 Concepts You Must Understand First

Before starting, verify you can answer these questions:

tmpfs and RAM Filesystems:

  • Q: What is tmpfs and how does it differ from ramfs?
  • A: tmpfs is a RAM-backed filesystem that can swap to disk. It has size limits. ramfs has no limits and never swaps.

Initramfs Purpose:

  • Q: What is initramfs and why is it needed?
  • A: An initial root filesystem in RAM that runs before the real root. It sets up devices, mounts filesystems, and hands off to the real init.

Overlay Filesystems:

  • Q: How does an overlay work conceptually?
  • A: An overlay combines multiple filesystem layers. Upper layers mask lower layers. Writes go to the upper layer.

Package Management:

  • Q: How does APK differ from apt?
  • A: APK is simpler, faster, uses “world” file for explicit package lists, supports virtual packages.

Init Systems:

  • Q: How does OpenRC differ from systemd?
  • A: OpenRC uses shell scripts, simpler dependency model, no journald, uses rc-service/rc-update commands.

5.5 Questions to Guide Your Design

Boot Configuration:

  1. Should the boot medium be FAT32 or ext4? Why?
  2. What kernel parameters are needed for diskless mode?
  3. How does initramfs find the apkovl file?

Persistence:

  1. What should be persisted (saved with LBU)?
  2. What should be ephemeral (reset on reboot)?
  3. How do you handle SSH host keys?

Kiosk Design:

  1. How do you prevent users from accessing the desktop?
  2. What happens if the kiosk application crashes?
  3. How do you update the kiosk remotely?

Resource Management:

  1. How much RAM is needed for your use case?
  2. What’s the trade-off between RAM size and package cache?
  3. How do you monitor resource usage?

5.6 Thinking Exercise

Before writing any code, mentally trace a boot sequence:

Phase 1: BIOS/UEFI (T=0)

  • BIOS loads syslinux from USB boot sector
  • Syslinux reads syslinux.cfg for kernel location
  • Syslinux loads vmlinuz-lts and initramfs-lts

Phase 2: Kernel + Initramfs (T=2s)

  • Kernel initializes, decompresses initramfs
  • Initramfs /init script runs
  • Scans for boot medium labeled “ALPINE”
  • Creates tmpfs, mounts boot medium read-only
  • Searches for *.apkovl.tar.gz

Phase 3: Root Switch (T=5s)

  • Extracts apkovl to tmpfs /etc
  • Reads /etc/apk/world
  • Installs packages from /media/usb/apks/
  • Switches root to tmpfs

Phase 4: OpenRC (T=10s)

  • Runs /sbin/init (OpenRC)
  • Executes sysinit runlevel (mount filesystems)
  • Executes boot runlevel (networking)
  • Executes default runlevel (services)

Phase 5: Kiosk (T=15s)

  • local service runs /etc/local.d/kiosk.start
  • Script starts X11 or Cage
  • Launches Chromium in kiosk mode
  • User sees application

Draw the Memory Map:

RAM (1GB example)
┌─────────────────────────────────────┐ 0x00000000
│ Reserved (BIOS, etc.)               │
├─────────────────────────────────────┤ ~0x00100000
│ Kernel (~10 MB)                     │
├─────────────────────────────────────┤
│ tmpfs root filesystem (~100 MB)     │
│ - /usr, /bin, /lib (packages)       │
│ - /etc (configuration)              │
│ - /var (runtime state)              │
├─────────────────────────────────────┤
│ Application memory (~200 MB)         │
│ - Chromium, X11, etc.               │
├─────────────────────────────────────┤
│ Buffer/Page cache (~600 MB)          │
│ - Available for file caching        │
│ - Reclaimed as needed               │
└─────────────────────────────────────┘ 0x40000000

5.7 Hints in Layers

Hint 1: Basic Diskless Setup

Start by creating a minimal diskless system:

# Boot from Alpine Extended ISO in QEMU
# At the login prompt:
localhost login: root

# Partition the virtual USB (/dev/vda)
apk add e2fsprogs dosfstools
fdisk /dev/vda
# Create one partition, mark bootable
# Type: 'n' for new, accept defaults, 'a' for bootable, 'w' to write

# Format as FAT32 (for BIOS compatibility) or ext4
mkfs.ext4 /dev/vda1

# Mount it
mkdir -p /media/usb
mount /dev/vda1 /media/usb

# Run Alpine setup in diskless mode
setup-alpine
# When asked for disk, choose: none (for diskless mode)
# When asked for LBU backup location: usb

# The setup creates your first apkovl

Hint 2: LBU Configuration

Configure LBU to save specific paths:

# View LBU configuration
cat /etc/lbu/lbu.conf

# Typical diskless configuration:
# LBU_MEDIA=usb
# LBU_BACKUPDIR=/media/usb

# See what will be saved
lbu status
# A = added, M = modified, D = deleted

# Include additional paths
lbu include /home/kiosk

# Exclude paths you don't want saved
lbu exclude /var/log

# Commit changes
lbu commit -d

Hint 3: APK Cache Setup

Populate the package cache for offline operation:

# Create cache directory on boot medium
mkdir -p /media/usb/apks

# Configure APK to use this cache
setup-apkcache /media/usb/apks

# Now install packages - they'll be cached
apk add chromium xorg-server xf86-video-fbdev

# Verify packages are cached
ls /media/usb/apks/

# Update cache index
apk update

# Make sure world file is saved
lbu commit

Hint 4: Auto-Login and Kiosk Startup

Create automatic login for the kiosk user:

# Create kiosk user
adduser -D -s /bin/sh kiosk

# Configure auto-login on tty1
# Edit /etc/inittab
vi /etc/inittab

# Change the tty1 line from:
# tty1::respawn:/sbin/getty 38400 tty1
# To:
tty1::respawn:/sbin/agetty -a kiosk tty1 linux

# Create kiosk startup script
mkdir -p /etc/local.d
cat > /etc/local.d/kiosk.start << 'EOF'
#!/bin/sh
# Start X11 on tty1 as kiosk user
su - kiosk -c 'startx' &
EOF
chmod +x /etc/local.d/kiosk.start

# Enable local service
rc-update add local default

# Save configuration
lbu commit

Hint 5: X11 Kiosk Configuration

Create the X11 startup script for the kiosk:

# As kiosk user (or create as root and fix permissions)
cat > /home/kiosk/.xinitrc << 'EOF'
#!/bin/sh

# Disable screen saver and power management
xset s off
xset -dpms
xset s noblank

# Hide cursor after 5 seconds of inactivity
unclutter -idle 5 &

# Start Chromium in kiosk mode
exec chromium-browser \
    --kiosk \
    --noerrdialogs \
    --disable-infobars \
    --no-first-run \
    --disable-translate \
    --disable-features=TranslateUI \
    --disk-cache-dir=/dev/null \
    --disk-cache-size=1 \
    http://your-dashboard-url.com
EOF

chmod +x /home/kiosk/.xinitrc
chown kiosk:kiosk /home/kiosk/.xinitrc

# Save to overlay
lbu include /home/kiosk
lbu commit

Hint 6: Complete Working Script

Here’s a comprehensive setup script:

#!/bin/sh
# setup-kiosk.sh - Run on a fresh Alpine diskless installation

set -e

# Variables - customize these
HOSTNAME="kiosk"
KIOSK_URL="http://dashboard.local"
SSH_PUBKEY="ssh-ed25519 AAAA... your-key-here"

echo "=== Setting up Alpine Diskless Kiosk ==="

# Set hostname
echo "$HOSTNAME" > /etc/hostname
hostname -F /etc/hostname

# Configure network (DHCP)
cat > /etc/network/interfaces << EOF
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp
EOF

# Install required packages
apk update
apk add \
    openssh \
    chromium \
    xorg-server \
    xf86-video-vesa \
    xf86-video-fbdev \
    xf86-input-libinput \
    xinit \
    ttf-dejavu \
    dbus \
    eudev \
    unclutter

# Configure SSH
rc-update add sshd default
mkdir -p /root/.ssh
echo "$SSH_PUBKEY" > /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys

# Create kiosk user
adduser -D -s /bin/sh kiosk
mkdir -p /home/kiosk/.ssh
echo "$SSH_PUBKEY" > /home/kiosk/.ssh/authorized_keys
chmod 600 /home/kiosk/.ssh/authorized_keys
chown -R kiosk:kiosk /home/kiosk

# Configure auto-login
sed -i 's|^tty1::respawn:/sbin/getty.*|tty1::respawn:/sbin/agetty -a kiosk -n tty1 linux|' /etc/inittab

# Create kiosk .profile
cat > /home/kiosk/.profile << 'EOF'
# Start X if on tty1 and not already running
if [ "$(tty)" = "/dev/tty1" ] && [ -z "$DISPLAY" ]; then
    exec startx
fi
EOF

# Create .xinitrc
cat > /home/kiosk/.xinitrc << EOF
#!/bin/sh
xset s off
xset -dpms
xset s noblank
unclutter -idle 5 &
exec chromium-browser \\
    --kiosk \\
    --noerrdialogs \\
    --disable-infobars \\
    --no-first-run \\
    ${KIOSK_URL}
EOF

chmod +x /home/kiosk/.profile /home/kiosk/.xinitrc
chown -R kiosk:kiosk /home/kiosk

# Enable services
rc-update add networking boot
rc-update add dbus default
rc-update add udev sysinit

# Include home directory in LBU
lbu include /home/kiosk

# Save configuration
echo "=== Saving configuration with LBU ==="
lbu commit -d

echo "=== Kiosk setup complete! ==="
echo "Reboot to test: reboot"

5.8 The Interview Questions They’ll Ask

Basic Understanding

  1. “What is Alpine’s diskless mode and why would you use it?”
    • Good Answer: Diskless mode runs the entire OS from RAM (tmpfs). The boot medium is read-only after boot. Use cases: kiosks (tamper-proof), embedded systems (no disk wear), firewalls (instant recovery), Raspberry Pi (SD card longevity).
    • Red Flag: “It’s like a live CD” (misses persistence with LBU)
  2. “How does configuration persistence work in diskless mode?”
    • Good Answer: LBU (Local Backup Utility) creates tarballs of /etc and other paths. These apkovl files are extracted at boot by initramfs. You must explicitly run lbu commit to save changes.
    • Red Flag: “Changes are automatically saved” (they’re not)
  3. “What’s the difference between sys, data, and diskless install modes?”
    • Good Answer:
      • sys: Traditional install to disk (like Ubuntu)
      • data: OS in RAM, /var on disk (hybrid)
      • diskless: Entire OS in RAM, configuration on boot medium
    • Red Flag: “I only know sys mode”

Technical Details

  1. “Walk me through Alpine’s diskless boot process.”
    • Good Answer: BIOS loads bootloader → loads kernel + initramfs → initramfs finds boot medium → creates tmpfs → extracts apkovl → installs packages from cache → switches root → OpenRC starts services.
    • Follow-up: “Where does the apkovl get extracted?” (To the tmpfs root before pivot_root)
  2. “How do you ensure the system boots without network access?”
    • Good Answer:
      1. Set up APK cache on boot medium
      2. Download all packages: apk update && apk cache download
      3. Ensure world file lists all needed packages
      4. Save with lbu commit
    • Red Flag: “Just install packages and reboot” (won’t work offline)
  3. “What happens if someone unplugs the USB while the system is running?”
    • Good Answer: The system continues running normally because it’s in RAM. However, you can’t run lbu commit (no place to save). On reboot, it reverts to the last saved state.
    • Follow-up: “How would you detect this situation?”
  4. “How do you handle SSH host keys in diskless mode?”
    • Good Answer: Generate once on first boot, then lbu commit to persist them. Or, generate deterministically from hardware serial. Or, accept regeneration (changes host fingerprint on reboot if not saved).
    • Red Flag: “They regenerate every boot” (security and convenience issues)

Problem-Solving

  1. “Your kiosk boots to a black screen. Debugging steps?”
    • Good Answer:
      1. Boot with serial console to see boot messages
      2. Check if packages installed (apk cache issue?)
      3. Check X11 logs in /var/log/Xorg.0.log
      4. Try running startx manually
      5. Verify video driver is correct
      6. Check .xinitrc syntax
  2. “Your diskless system is running out of RAM. What do you do?”
    • Good Answer:
      1. Check what’s consuming RAM: free -h, ps aux --sort=-%mem
      2. Reduce tmpfs size in initramfs (smaller / means less overhead)
      3. Remove unnecessary packages from world file
      4. Use data mode (put /var on disk)
      5. Add swap file (on disk, defeats some diskless benefits)
  3. “How would you deploy 100 identical kiosks?”
    • Good Answer:
      1. Create master image with all configuration
      2. Clone boot medium (dd or balenaEtcher)
      3. On first boot, each kiosk generates unique hostname, SSH keys
      4. Use DHCP/DNS for network config
      5. Remote management via SSH or Ansible
      6. Updates: SSH in, apk upgrade, lbu commit, reboot

5.9 Books That Will Help

Topic Book Specific Chapter/Section Why It Helps
Linux Boot Process “How Linux Works” by Brian Ward Chapter 5: How the Linux Kernel Boots Deep dive into kernel, initramfs, init
Embedded Linux “Mastering Embedded Linux Programming” by Chris Simmonds Chapter 4: Configuring and Building the Kernel initramfs, root filesystems
System Administration “The Linux Programming Interface” by Michael Kerrisk Chapter 27: Program Execution How init and exec work
Shell Scripting “The Linux Command Line” by William Shotts Chapters 24-30: Shell Scripts Writing robust init scripts
tmpfs/ramfs “Linux Kernel Development” by Robert Love Chapter 16: The Page Cache How tmpfs uses memory

Online Resources:

5.10 Implementation Phases

Phase 1: Basic Diskless Boot (4 hours)

  • Download Alpine Extended ISO
  • Boot in QEMU with virtual USB
  • Run setup-alpine in diskless mode
  • Verify boot from USB works
  • Milestone: System boots to shell from USB

Phase 2: LBU Understanding (4 hours)

  • Make configuration changes
  • Understand lbu status output
  • Practice lbu commit
  • Test reboot preserves changes
  • Milestone: Hostname persists across reboots

Phase 3: Package Cache (4 hours)

  • Set up APK cache on boot medium
  • Install packages
  • Test offline boot
  • Milestone: System boots without network

Phase 4: Kiosk Application (8 hours)

  • Install X11 and Chromium
  • Create kiosk user
  • Configure auto-login
  • Create startup scripts
  • Milestone: Browser launches automatically

Phase 5: Hardening (6 hours)

  • Configure SSH key-only access
  • Remove unnecessary packages
  • Test power-loss recovery
  • Create documentation
  • Milestone: Production-ready system

Phase 6: Raspberry Pi Adaptation (8 hours) (Optional)

  • Download Alpine for aarch64
  • Understand Pi boot process (different from x86)
  • Configure hardware-specific settings
  • Test on real hardware
  • Milestone: Working Pi kiosk

5.11 Key Implementation Decisions

  1. FAT32 vs ext4 for boot medium: FAT32 for maximum compatibility (BIOS, UEFI, Pi), but 4GB file limit. ext4 for larger files, but may need UEFI or ext-aware bootloader.

  2. Syslinux vs GRUB: Syslinux is simpler for FAT32 boot. GRUB needed for advanced features (UEFI Secure Boot, complex menus).

  3. X11 vs Cage/Wayland: X11 has more compatibility. Cage (Wayland compositor) is simpler for single-app kiosks. Choose based on application needs.

  4. Chromium vs Firefox: Chromium has better kiosk mode (--kiosk). Firefox needs extensions for true kiosk mode. Consider memory usage.

  5. Auto-login method: Modify /etc/inittab for agetty auto-login, or use a display manager like LightDM with autologin. inittab is simpler.

  6. SSH host key handling:

    • Option A: Generate and save with LBU (fingerprint changes if overlay lost)
    • Option B: Derive from hardware serial (same key every time)
    • Option C: Allow regeneration (simpler but inconvenient)

6. Testing Strategy

6.1 Unit Testing

Test each component individually:

LBU Functionality:

# Test: Changes are detected
echo "test" > /etc/test.txt
lbu status | grep test.txt
# Expected: "A etc/test.txt"

# Test: Commit works
lbu commit
ls /media/usb/*.apkovl.tar.gz
# Expected: hostname.apkovl.tar.gz exists

# Test: Restoration works
rm /etc/test.txt
reboot
# After reboot:
cat /etc/test.txt
# Expected: "test"

APK Cache:

# Test: Packages are cached
apk add htop
ls /media/usb/apks/ | grep htop
# Expected: htop-*.apk

# Test: Offline installation
# (disconnect network in QEMU: -nic none)
apk del htop
apk add htop
# Expected: Installs from cache

Auto-Login:

# Test: kiosk user logs in automatically
# Reboot, observe console
# Expected: No password prompt, kiosk user logged in

6.2 Integration Testing

# Test 1: Cold boot (power off, power on)
poweroff
# Start QEMU again
# Expected: System boots, kiosk appears

# Test 2: Reboot cycle
for i in 1 2 3 4 5; do
    reboot
    sleep 60  # Wait for boot
    # Check system is up
    ssh kiosk@kiosk-ip "echo 'Reboot $i OK'"
done
# Expected: All 5 reboots succeed

# Test 3: Power loss simulation
# In QEMU: Ctrl+C to kill (simulates power loss)
# Restart QEMU
# Expected: System recovers, no corruption

# Test 4: Network disconnection
# Boot with network, then disconnect
# Run kiosk overnight
# Expected: No crashes or errors

6.3 Debugging Techniques

Serial Console:

# Add console output to see boot messages
qemu-system-x86_64 \
    -drive file=usb.img,format=raw \
    -serial stdio \
    -nographic

# Add to kernel params in syslinux.cfg:
# console=ttyS0,115200

Boot Logging:

# View boot logs (from running system)
dmesg | less

# View apk installation log
cat /var/log/apk.log

# View service status
rc-status -a

QEMU Monitor:

# Start with monitor
qemu-system-x86_64 -monitor stdio ...

# Useful commands:
(qemu) info block     # Show block devices
(qemu) info network   # Show network
(qemu) system_powerdown  # Clean shutdown

7. Common Pitfalls & Debugging

Problem 1: “Boot hangs at ‘Loading apkovl…’“

  • Root Cause: Corrupted or missing apkovl file
  • Fix:
    1. Boot without apkovl: remove *.apkovl.tar.gz from boot medium
    2. Reconfigure from scratch
    3. Save new apkovl
  • Quick Test: tar -tzf hostname.apkovl.tar.gz should list files

Problem 2: “Packages not installed after reboot”

  • Root Cause: APK cache not configured or packages not in world file
  • Fix:
    1. Verify cache: ls /media/usb/apks/
    2. Verify world: cat /etc/apk/world
    3. Rebuild cache: apk update && apk cache download
    4. Commit: lbu commit
  • Quick Test: apk add --simulate package should show “Using cached…”

Problem 3: “No space left on device”

  • Root Cause: tmpfs is full (RAM limit)
  • Fix:
    1. Check usage: df -h /
    2. Clear logs: rm -rf /var/log/*
    3. Reduce package count
    4. Increase RAM allocation
  • Quick Test: free -h shows available memory

Problem 4: “Kiosk starts then immediately exits”

  • Root Cause: X11 or application crash
  • Fix:
    1. Check X11 log: cat /var/log/Xorg.0.log
    2. Try startx manually to see errors
    3. Install correct video driver
    4. Check .xinitrc syntax
  • Quick Test: DISPLAY=:0 chromium-browser (run from SSH)

Problem 5: “Changes lost after reboot”

  • Root Cause: Forgot to run lbu commit
  • Fix: This is expected behavior! Always lbu commit after changes.
  • Quick Test: lbu status shows pending changes

Problem 6: “Cannot mount boot medium after boot”

  • Root Cause: Boot medium is read-only after diskless boot
  • Fix: This is expected! To write:
    1. mount -o remount,rw /media/usb
    2. Make changes
    3. lbu commit
    4. mount -o remount,ro /media/usb
  • Quick Test: mount | grep media shows mount options

Problem 7: “Different hostname on each boot”

  • Root Cause: Hostname not saved in apkovl
  • Fix:
    1. echo "kiosk" > /etc/hostname
    2. lbu commit
  • Quick Test: Hostname is in lbu list

Problem 8: “SSH host key changes on reboot”

  • Root Cause: Host keys not saved or regenerated
  • Fix:
    1. lbu include /etc/ssh
    2. lbu commit
  • Quick Test: Connect with SSH, reboot, connect again (no warning)

8. Extensions & Challenges

Extension 1: Watchdog Timer

  • Use hardware watchdog to auto-reboot on freeze
  • apk add watchdog
  • Configure to reboot if system hangs
  • Challenge: Handle graceful shutdown

Extension 2: Network Boot (PXE)

  • Boot diskless Alpine over network
  • No local storage needed at all
  • Centralized image management
  • Challenge: TFTP and DHCP configuration

Extension 3: Read-Only with Overlay

  • Use OverlayFS for selective writes
  • Some paths read-only, some writable
  • Challenge: Configure complex overlay mount

Extension 4: Encrypted Boot Medium

  • Encrypt the apkovl for sensitive deployments
  • Password or TPM unlock at boot
  • Challenge: Key management

Extension 5: Multi-Kiosk Management

  • Ansible playbook for fleet management
  • Centralized configuration
  • Rolling updates
  • Challenge: Handle network failures

Extension 6: Raspberry Pi Hardware

  • Adapt for real Pi hardware
  • GPIO integration
  • Pi-specific video drivers
  • Challenge: Different boot process

Extension 7: Automatic Updates

  • Script to pull updates, apply, lbu commit
  • Rollback on failure
  • Challenge: Atomic updates

Extension 8: Monitoring and Alerting

  • Lightweight monitoring (node_exporter)
  • Send alerts on issues
  • RAM usage tracking
  • Challenge: Minimal resource usage

9. Real-World Connections

Chromebooks (ChromeOS)

  • Similar read-only root with verified boot
  • User data persists, system doesn’t
  • Factory reset = instant

Thin Clients (HP, Dell, IGEL)

  • Boot from network or local flash
  • Run entirely from RAM
  • Centrally managed images

Network Appliances (pfSense, OPNsense)

  • Based on BSD, similar concept
  • RAM-based operation
  • Configuration saved to XML

Point-of-Sale Systems

  • Read-only for tamper resistance
  • Known-good state on reboot
  • Difficult to compromise

Digital Signage (BrightSign, screenly)

  • Minimal OS for stability
  • Auto-recovery on corruption
  • Remote management

IoT Gateways

  • SD card longevity critical
  • Power-loss resilience
  • Secure boot requirements

10. Resources

Primary References

Technical Documentation

Community Resources

Example Projects


11. Self-Assessment Checklist

Before considering this project complete, verify:

Understanding:

  • Can you explain what diskless mode is to a colleague?
  • Can you describe the boot process from BIOS to kiosk?
  • Do you understand the difference between sys, data, and diskless modes?
  • Can you explain how LBU and apkovl work?
  • Do you know when to use diskless mode vs traditional install?

Implementation:

  • Can you set up diskless mode from scratch without referring to hints?
  • Can you configure LBU to save custom paths?
  • Can you set up APK cache for offline operation?
  • Can you create a kiosk that auto-starts on boot?
  • Can you SSH into the kiosk for remote management?

Debugging:

  • Can you recover from a corrupted apkovl?
  • Can you diagnose boot failures using serial console?
  • Can you identify why packages aren’t installing?
  • Can you fix a kiosk that shows a black screen?

Production Readiness:

  • Does your system boot without network access?
  • Does your system recover cleanly from power loss?
  • Have you tested the reboot cycle multiple times?
  • Is SSH access configured with key-based auth?
  • Have you documented your specific configuration?

12. Submission / Completion Criteria

Your implementation is complete when:

Core Requirements:

  • System boots from USB/SD into RAM (verified with mount | grep "on / " showing tmpfs)
  • Boot medium is read-only after boot (verified with mount | grep media)
  • Configuration persists across reboots via LBU
  • System boots successfully without network access
  • Kiosk application launches automatically
  • SSH access works for remote management

Testing:

  • Successfully completed 5 consecutive reboot cycles
  • Recovered from simulated power loss (QEMU kill)
  • Verified APK cache enables offline operation
  • Tested LBU backup and restore

Documentation:

  • Created setup script or documented manual steps
  • Documented all customizations made
  • Listed all installed packages
  • Noted any decisions and trade-offs

Optional Challenges:

  • Adapted for Raspberry Pi (if targeting ARM)
  • Set up network boot (PXE)
  • Created multi-kiosk deployment script
  • Implemented watchdog for reliability

Congratulations! You’ve built a production-ready diskless kiosk system. This project taught you Alpine’s unique installation mode, the LBU persistence mechanism, and how to create tamper-resistant embedded systems. These skills apply directly to IoT devices, firewalls, digital signage, and any scenario where system reliability and predictability are paramount.

Next Steps:

  • Extend to multiple kiosks with centralized management
  • Add monitoring and alerting
  • Explore PXE network boot for datacenter deployments
  • Consider adapting for specific hardware (Raspberry Pi, Intel NUC)