Project 9: Jail Lab (Classic + VNET)

Build classic and VNET jails, run isolated services, and prove network separation.

Quick Reference

Attribute Value
Difficulty Level 3
Time Estimate 2-3 weekends
Main Programming Language Shell (sh) (Alternatives: csh, Python)
Alternative Programming Languages sh, csh, Python
Coolness Level Level 4
Business Potential Level 2
Prerequisites Projects 1-8 complete, ZFS preferred
Key Topics jails, jail.conf, VNET, virtual interfaces

1. Learning Objectives

By completing this project, you will:

  1. Create a classic jail and run a service inside it.
  2. Create a VNET jail with its own network stack.
  3. Configure networking for both types and validate isolation.
  4. Document jail layout and management workflow.

2. All Theory Needed (Per-Concept Breakdown)

Concept 1: FreeBSD Jails (Classic vs VNET)

Fundamentals Jails are FreeBSD’s lightweight OS-level isolation mechanism. A classic jail shares the host’s network stack and uses IP addresses assigned on the host. A VNET jail has its own virtual network stack and interfaces, providing stronger isolation and more flexibility. Both types use a separate filesystem root and their own process namespace. For this project, you must understand the difference between classic and VNET jails, when to use each, and how they relate to services and security.

In practice, write a short checklist for FreeBSD jail isolation and confirm it after each reboot. This keeps the concept concrete and prevents accidental drift between sessions.

Deep Dive into the concept FreeBSD jails isolate processes, filesystem views, and, optionally, networking. In classic jails, the host assigns an IP address and the jail uses the host’s network stack. This is simpler and requires fewer moving parts, making it ideal for quick service isolation. However, it also means the jail shares the host’s routing and network configuration. VNET jails, by contrast, provide a virtual network stack per jail, including its own interfaces, routing table, and firewall rules. This makes VNET more like a container with its own network namespace.

The jail lifecycle is managed via jail or the service jail wrapper, and configuration is centralized in /etc/jail.conf (or /etc/jail.conf.d). A jail’s filesystem is typically built from a base system extraction into a directory like /jails/nginx. Because the base system and packages are separate, you can create “thick” jails (full base + packages) or “thin” jails (shared read-only base with overlays). For this project, a thick jail is sufficient and easier to reason about.

Networking for classic jails is defined by ip4.addr in jail.conf. For VNET, you must create a virtual interface (typically epair), attach one side to a bridge on the host, and move the other into the jail. The jail then configures its own interface and IP. This is more complex but gives you a realistic isolation model and teaches you the plumbing that underlies containers.

Jails are powerful because they let you run multiple services with reduced overhead compared to VMs. They are used in production for hosting, CI, and service isolation. The key is to maintain clear boundaries: each jail should have a defined purpose, a minimal package set, and documented network access. This project builds the operational habits needed to manage jails safely.

Operationally, FreeBSD jail isolation is easiest to keep stable when you treat it as a small contract between configuration, tooling, and observable outputs. Write down the exact files that own the state and the commands that reveal the current truth. Then verify the contract at three points: immediately after you make the change, after a reboot, and after a deliberate disturbance such as restarting services or reloading modules. FreeBSD rewards this discipline because it rarely hides state; if something changes, it is usually in a file you control. Make a habit of collecting a before-and-after snapshot of commands and outputs so you can explain which change caused which effect.

At scale, FreeBSD jail isolation is also about failure containment. Identify what must remain available when something breaks and design a safe escape hatch. For example, keep console access for firewall changes, keep a previous boot environment for upgrades, or keep a dataset snapshot before risky edits. The same pattern applies across domains: define invariants, define the rollback path, and then only proceed when you can trigger that rollback quickly. Finally, test the failure path while the system is healthy; you learn more from a controlled rollback than from an emergency. This perspective turns the lab exercise into an operational capability you can trust on production systems.

How this fits on projects

Definitions & key terms

  • Jail -> Process and filesystem isolation mechanism.
  • Classic jail -> Shares host network stack.
  • VNET jail -> Separate virtual network stack.
  • epair -> Virtual Ethernet pair interface.

Mental model diagram

Host
+-- classic jail (shared net)
+-- vnet jail (virtual net stack)

How it works (step-by-step, with invariants and failure modes)

  1. Create jail root filesystem.
  2. Define jail in jail.conf.
  3. Start jail and verify processes.
  4. Configure networking (classic or VNET).

Invariants:

  • Jail root must contain a base system.
  • jail.conf must define name and path.

Failure modes:

  • Missing base system -> jail fails to start.
  • Misconfigured network -> jail unreachable.

Minimal concrete example

jls
jexec 1 /bin/sh

Common misconceptions

  • “Jails are the same as VMs.” -> Jails share the host kernel.
  • “VNET is always required.” -> Classic jails are simpler and sufficient in many cases.

Check-your-understanding questions

  1. What is the main difference between classic and VNET jails?
  2. Why do jails require their own root filesystem?
  3. Which tool lists running jails?

Check-your-understanding answers

  1. VNET has its own network stack; classic jails share the host’s.
  2. To isolate filesystem view and prevent access to host paths.
  3. jls.

Real-world applications

  • Multi-tenant hosting with lightweight isolation.
  • Running multiple services on a single FreeBSD host.

Where you’ll apply it

  • Section 3.2 Functional Requirements
  • Section 5.10 Phase 2
  • Also used in: P10 Update Runbook

References

  • “Absolute FreeBSD, 3rd Edition” (Ch. 22)
  • FreeBSD Handbook: Jails

Key insights Jails isolate services without the overhead of full virtualization.

Summary Classic jails are simple; VNET jails are powerful. Both are essential FreeBSD skills.

Homework/Exercises to practice the concept

  1. Create a classic jail and enter it with jexec.
  2. Compare processes inside and outside the jail.
  3. Document which files are visible inside the jail.

Solutions to the homework/exercises

  1. Use jail.conf and service jail start.
  2. ps shows only jail processes.
  3. The jail sees only its root filesystem.

Concept 2: Jail Networking and Service Exposure

Fundamentals Jail networking defines how services inside jails are reached. Classic jails use host-assigned IPs, while VNET jails create their own interfaces. Both require careful mapping of IP addresses, routes, and firewall rules. A service inside a jail should be reachable only on intended interfaces. For this project, you must configure IP addressing, validate connectivity, and confirm that services in separate jails do not conflict.

In practice, write a short checklist for jail networking and confirm it after each reboot. This keeps the concept concrete and prevents accidental drift between sessions.

In practice, rehearse the steps on a disposable VM so you can recognize normal outputs and failure signals quickly.

Deep Dive into the concept In classic jails, networking is straightforward: you assign an IP in jail.conf (ip4.addr) and the jail can bind to it. The host must have that IP configured on its interface, and you typically use IP aliases to support multiple jails. Because the host and jail share the network stack, firewalling is done on the host, and services are visible on the host interface. This is simpler but less isolated.

VNET jails use virtual interfaces, commonly created as epair devices. An epair creates two connected interfaces: epair0a and epair0b. You attach epair0a to a bridge on the host and move epair0b into the jail. The jail then configures its own IP and routes. This gives it an independent network stack, which is closer to container networking. It also means you must manage bridges and ensure that the host forwards traffic correctly. This is a significant jump in complexity, but it teaches real network isolation.

Service exposure must be explicit. If you run a web server in a jail, you should verify it listens only on the jail’s IP, not on the host. You should also avoid port conflicts by ensuring each jail uses a distinct IP. In classic jails, this is mandatory because they share the host stack. In VNET, you can reuse ports because each jail has its own stack, but you still must plan IPs carefully.

Troubleshooting jail networking follows the same layered approach as host networking: check interface status, check IP, check route, check firewall, and check service bindings. Use jexec to run ifconfig and netstat -rn inside the jail. This is where the value of VNET becomes clear: the jail has its own view of the network, which you can inspect independently of the host.

Operationally, jail networking is easiest to keep stable when you treat it as a small contract between configuration, tooling, and observable outputs. Write down the exact files that own the state and the commands that reveal the current truth. Then verify the contract at three points: immediately after you make the change, after a reboot, and after a deliberate disturbance such as restarting services or reloading modules. FreeBSD rewards this discipline because it rarely hides state; if something changes, it is usually in a file you control. Make a habit of collecting a before-and-after snapshot of commands and outputs so you can explain which change caused which effect.

At scale, jail networking is also about failure containment. Identify what must remain available when something breaks and design a safe escape hatch. For example, keep console access for firewall changes, keep a previous boot environment for upgrades, or keep a dataset snapshot before risky edits. The same pattern applies across domains: define invariants, define the rollback path, and then only proceed when you can trigger that rollback quickly. Finally, test the failure path while the system is healthy; you learn more from a controlled rollback than from an emergency. This perspective turns the lab exercise into an operational capability you can trust on production systems.

How this fits on projects

  • This concept is used in Section 3.2 and Section 5.10.
  • It builds directly on P05 Network Lab.

Definitions & key terms

  • epair -> Virtual Ethernet pair for VNET.
  • bridge -> Virtual switch on the host.
  • ip4.addr -> jail.conf IP assignment.

Mental model diagram

Host bridge -> epair0a -- epair0b -> VNET jail

How it works (step-by-step, with invariants and failure modes)

  1. Create epair and bridge on host.
  2. Attach one end to bridge, move the other to jail.
  3. Configure IP inside jail.
  4. Start service and test connectivity.

Invariants:

  • epair ends must stay paired.
  • Jail must have a default route.

Failure modes:

  • No bridge -> jail has no LAN access.
  • Wrong IP -> service unreachable.

Minimal concrete example

ifconfig epair0a create
ifconfig bridge0 addm epair0a

Common misconceptions

  • “Classic jails can use the same port on the same IP.” -> They cannot.
  • “VNET is optional for isolation.” -> It provides stronger separation.

Check-your-understanding questions

  1. Why do VNET jails need epair interfaces?
  2. How do you check routes inside a jail?
  3. Why must classic jails use unique IPs?

Check-your-understanding answers

  1. They need a virtual NIC to attach to the jail’s network stack.
  2. jexec 1 netstat -rn.
  3. They share the host stack; ports would conflict.

Real-world applications

  • Multi-service hosts with isolated networking.
  • Hosting environments with per-jail firewalls.

Where you’ll apply it

  • Section 3.2 Functional Requirements
  • Section 5.10 Phase 2
  • Also used in: P05 Network Lab

References

  • “Absolute FreeBSD, 3rd Edition” (Ch. 22)
  • FreeBSD Handbook: VNET Jails

Key insights Jail networking is the difference between “lightweight process isolation” and “real network isolation.”

Summary Classic jails are simple; VNET jails require more setup but provide stronger isolation.

Homework/Exercises to practice the concept

  1. Create an epair and bridge, then delete them.
  2. Assign an IP alias on the host for a classic jail.
  3. Verify a jail’s route table.

Solutions to the homework/exercises

  1. ifconfig epair0a destroy cleans up.
  2. ifconfig em0 alias 192.168.1.60/24.
  3. jexec 1 netstat -rn.

3. Project Specification

3.1 What You Will Build

Two jails: a classic jail with a service bound to a host IP alias, and a VNET jail with its own virtual network stack and IP address. Each jail runs a lightweight service and is reachable only on its assigned IP.

3.2 Functional Requirements

  1. Classic jail created and running.
  2. VNET jail created with epair and bridge.
  3. Services running inside each jail.
  4. Distinct IPs assigned and reachable.
  5. Isolation verified with jls and network tests.

3.3 Non-Functional Requirements

  • Performance: Jail startup completes in under 30 seconds.
  • Reliability: Jails start after reboot.
  • Usability: jail.conf is documented and readable.

3.4 Example Usage / Output

$ jls
JID IP Address Hostname Path
1 192.168.1.60 webjail /jails/web
2 192.168.1.70 vnetjail /jails/vnet

3.5 Data Formats / Schemas / Protocols

  • jail.conf snippet 
    webjail {
 path = /jails/web;
 host.hostname = webjail;
 ip4.addr = 192.168.1.60;
}

3.6 Edge Cases

  • Jail fails to start because base system missing.
  • VNET jail cannot reach LAN due to missing bridge.
  • Port conflicts in classic jails.

3.7 Real World Outcome

You can run isolated services with distinct IPs and understand their network separation.

3.7.1 How to Run (Copy/Paste)

service jail start webjail
service jail start vnetjail
jls

3.7.2 Golden Path Demo (Deterministic)

  • Use fixed IPs and jail names.
  • Record outputs without timestamps.

3.7.3 If CLI: provide an exact terminal transcript

$ jls
JID IP Address Hostname Path
1 192.168.1.60 webjail /jails/web
2 192.168.1.70 vnetjail /jails/vnet

$ echo $?
0

Failure demo (deterministic)

$ service jail start missingjail
jail: unknown jail: missingjail

$ echo $?
1

4. Solution Architecture

4.1 High-Level Design

Host ZFS datasets -> jail roots -> jail.conf -> classic + VNET jails

4.2 Key Components

Component Responsibility Key Decisions
Jail roots Filesystem isolation Separate datasets
jail.conf Jail definitions Classic vs VNET
Network bridge VNET connectivity bridge0 with epair

4.3 Data Structures (No Full Code)

JailPlan
- webjail: classic, ip 192.168.1.60
- vnetjail: vnet, ip 192.168.1.70

4.4 Algorithm Overview

Key Algorithm: Jail Bring-Up

  1. Build jail root filesystem.
  2. Define jail.conf entries.
  3. Configure network (alias or VNET).
  4. Start jails and verify services.

Complexity Analysis:

  • Time: O(jail creation + package install)
  • Space: O(jail root size)

5. Implementation Guide

5.1 Development Environment Setup

# Ensure ZFS datasets exist for jail roots

5.2 Project Structure

jail-lab/
+-- jail.conf
+-- network-setup.sh
+-- notes.md

5.3 The Core Question You’re Answering

“How can I isolate services without full VMs?”

5.4 Concepts You Must Understand First

Stop and research these before coding:

  1. Jails (classic vs VNET)
  2. Jail networking and service exposure

5.5 Questions to Guide Your Design

  1. Where will you store jail roots?
  2. Which services will each jail run?
  3. How will you assign IPs to avoid conflicts?

5.6 Thinking Exercise

Containment Exercise

List which host filesystems and devices a jail should NOT access.

5.7 The Interview Questions They’ll Ask

  1. What is the main difference between a jail and a VM?
  2. How does VNET change isolation?
  3. How do you manage services inside a jail?
  4. What are thin vs thick jails?
  5. What are common jail networking mistakes?

5.8 Hints in Layers

Hint 1: Build a classic jail first Start with simplicity.

Hint 2: Add VNET after success Reuse patterns for configuration.

Hint 3: Use jail.conf Keep definitions centralized.

Hint 4: Validate with jls and ifconfig Ensure correct IPs and jail IDs.

5.9 Books That Will Help

Topic Book Chapter
Jails “Absolute FreeBSD, 3rd Edition” Ch. 22

5.10 Implementation Phases

Phase 1: Classic Jail (3-4 hours)

Goals: Create basic jail and service. Tasks:

  1. Build jail root.
  2. Define jail.conf and start jail. Checkpoint: jls shows classic jail.

Phase 2: VNET Jail (4-6 hours)

Goals: Build VNET jail networking. Tasks:

  1. Create epair and bridge.
  2. Start VNET jail and configure IP. Checkpoint: VNET jail reachable.

Phase 3: Service Isolation (3-4 hours)

Goals: Run services in both jails. Tasks:

  1. Install services inside jails.
  2. Test from host and LAN. Checkpoint: services reachable only on assigned IPs.

5.11 Key Implementation Decisions

Decision Options Recommendation Rationale
Jail type classic, VNET both Learn differences
Storage ZFS datasets, directories ZFS datasets Snapshot and rollback
Networking alias, epair+bridge both Practice both models

6. Testing Strategy

6.1 Test Categories

Category Purpose Examples
Jail Tests Running state jls
Network Tests Reachability ping, curl
Isolation Tests Host vs jail jexec checks

6.2 Critical Test Cases

  1. Classic jail start: appears in jls with IP.
  2. VNET jail network: has its own interface.
  3. Service reachability: each service reachable only on jail IP.

6.3 Test Data

Classic jail IP: 192.168.1.60
VNET jail IP: 192.168.1.70

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

Pitfall Symptom Solution
Missing base in jail Jail won’t start Populate jail root
No bridge for VNET No network in jail Create bridge0
IP conflicts Service unreachable Choose unique IPs

7.2 Debugging Strategies

  • Inspect from inside jail: jexec 1 ifconfig.
  • Check host bridge: ifconfig bridge0.

7.3 Performance Traps

  • Large numbers of jails without resource limits can exhaust memory.

8. Extensions & Challenges

8.1 Beginner Extensions

  • Add a second classic jail.
  • Snapshot jail dataset before changes.

8.2 Intermediate Extensions

  • Convert classic jail to VNET.
  • Use separate routing table inside jail.

8.3 Advanced Extensions

  • Integrate PF rules per jail.
  • Build a “jail template” for reuse.

9. Real-World Connections

9.1 Industry Applications

  • Multi-tenant hosting: low-overhead isolation.
  • CI environments: rapid test isolation.
  • iocage: jail management tool.
  • bastillebsd: jail automation system.

9.3 Interview Relevance

  • Explaining jails vs VMs.
  • Understanding network namespaces.

10. Resources

10.1 Essential Reading

  • “Absolute FreeBSD, 3rd Edition” by Michael W. Lucas - Ch. 22
  • FreeBSD Handbook - Jails

10.2 Video Resources

  • “FreeBSD Jails 101” - community tutorials

10.3 Tools & Documentation

  • jail(8): jail management manual
  • jail.conf(5): configuration reference

11. Self-Assessment Checklist

11.1 Understanding

  • I can explain classic vs VNET jails.
  • I can configure jail networking.
  • I can run services inside jails.

11.2 Implementation

  • Classic jail running.
  • VNET jail running.
  • Services reachable on jail IPs.

11.3 Growth

  • I documented jail layout decisions.
  • I can debug jail networking issues.
  • I can teach the jail workflow.

12. Submission / Completion Criteria

Minimum Viable Completion:

  • Classic jail running with service.
  • VNET jail created with network isolation.

Full Completion:

  • Both jails reachable on distinct IPs.
  • Jail configuration documented.

Excellence (Going Above & Beyond):

  • Jail templates or automation script created.
  • Per-jail firewall rules added.