Project 1: Plugin System for a Simple Shell

Build a mini shell that loads and unloads third-party plugins safely with a stable ABI and explicit lifecycle contracts.

Quick Reference

Attribute Value
Difficulty Intermediate
Time Estimate 1-2 weeks
Main Programming Language C (Alternatives: Rust via FFI, C++)
Alternative Programming Languages Rust, C++
Coolness Level Level 7 - extensible runtime system
Business Potential Level 6 - plugin architecture foundation
Prerequisites Shared libraries, function pointers, Makefiles
Key Topics dlopen/dlsym, ABI stability, lifecycle design

1. Learning Objectives

By completing this project, you will:

  1. Design a minimal plugin ABI that can evolve without breaking old plugins.
  2. Implement dynamic loading with dlopen/dlsym and clear error reporting.
  3. Define ownership rules for plugin state and command registration.
  4. Enforce compatibility checks (version + struct size + capabilities).
  5. Build a safe unload path that prevents stale function pointers.

2. All Theory Needed (Per-Concept Breakdown)

This section includes every concept required to implement the plugin system safely and predictably.

2.1 Dynamic Loading & Symbol Resolution

Fundamentals

Dynamic loading is how a process loads code at runtime instead of at link time. On POSIX systems, dlopen maps a shared object into your process, dlsym resolves a symbol address, and dlclose unloads it when no longer needed. These calls are your boundary to untrusted code. You must treat them as a contract: you only call a function pointer if you verified the symbol exists, the ABI matches, and the plugin is still loaded. The key idea is that symbols are looked up by name, not by type, so type safety is your responsibility. This makes careful header design and strict runtime validation essential. You should also understand that a loaded plugin can access your process memory, so the loader must defend against version mismatches and undefined behavior.

Deep Dive into the Concept

Dynamic loading is fundamentally about deferring linkage. At compile time, the host does not know which plugins will exist; at runtime, it must discover and validate them. This means the host becomes its own linker. When you call dlopen, the dynamic loader maps the .so into your process, performs relocations, and resolves dependencies. That mapping is process-global: once loaded, the plugin’s code shares address space with your host. Any ABI mismatch becomes a crash rather than a compile error. For example, if the host expects a struct plugin_api with fields (version, size, init, run, shutdown), and the plugin compiled against an older header that only has (version, init, run), then reading size will access the wrong memory. That leads to undefined behavior. This is why a two-field compatibility check (version + size) is the simplest safety net: it allows you to detect older layouts and refuse to load the plugin before you call into it.

Symbol resolution is also inherently unsafe if you skip checks. dlsym returns a void* which you cast to a function pointer. The cast compiles even if the actual symbol has a different signature. The only safe way is to ensure the plugin implements the exact function type specified in your public header and to keep that header stable. Another subtlety is symbol visibility and collisions. If you load with RTLD_GLOBAL, plugin symbols can leak into the global namespace and satisfy future unresolved symbols in other plugins. That can accidentally bind one plugin against another’s internal symbols. To avoid this, load with RTLD_LOCAL and only call functions you explicitly resolve.

You must also treat dlopen errors as part of your UX. dlerror() is a thread-local error string that is overwritten on each call. Proper usage requires clearing it before a call and reading it immediately after. A clean interface will wrap dlopen/dlsym errors into your own error model (e.g., PLUGIN_ERR_MISSING_SYMBOL) while preserving the raw dlerror() string for debugging.

Another deep issue is lifecycle. A plugin may register callbacks into the host. If you call dlclose while those callbacks are still stored, you now have dangling function pointers. Your host must define a strict lifecycle: load -> init -> register -> use -> unregister -> shutdown -> unload. The host should refuse to unload a plugin that still has active commands, or it must revoke them first. A safe policy is: unregister all commands as part of unload, then call shutdown, then dlclose. This is also a concurrency boundary: if commands can execute concurrently, you must synchronize unload to prevent a thread from calling into code that is being unloaded.

Finally, you should understand the difference between RTLD_NOW and RTLD_LAZY. RTLD_NOW resolves all symbols at dlopen time, failing fast if dependencies are missing. RTLD_LAZY resolves only when a symbol is first used, which can delay failures until later. For a plugin system, RTLD_NOW is usually safer: you want to detect missing dependencies at load time, not after a user runs a command. A production-grade plugin system often uses RTLD_NOW | RTLD_LOCAL by default, with optional overrides for advanced users.

How This Fits in This Project

You will implement a loader that scans a plugin directory, loads .so files, resolves a required entry symbol (plugin_get_api), validates version/size, and registers commands. You will also implement clean unloading and error reporting. See Sec. 4.1 for the loader architecture and Sec. 5.10 Phase 2 for the integration steps. You’ll apply the same dynamic-boundary discipline in Project 5’s middleware loading. Also used in: Project 5.

Definitions & Key Terms

  • Dynamic loader -> Runtime component that maps shared objects into a process.
  • dlopen -> Loads a shared object and returns a handle.
  • dlsym -> Resolves a symbol address from a handle.
  • dlclose -> Decrements refcount and unloads when unused.
  • RTLD_LOCAL -> Symbols are not made globally available.
  • RTLD_NOW -> Resolve all symbols at load time.

Mental Model Diagram (ASCII)

Host Process
+------------------+
| main()           |
| plugin manager   |
|  dlopen()        |----> [plugin.so mapped]
|  dlsym()         |----> plugin_get_api()
|  call api->init  |
+------------------+
           |
           v
   function pointers

How It Works (Step-by-Step)

  1. Host builds absolute path to a .so file.
  2. Host calls dlopen(path, RTLD_NOW | RTLD_LOCAL).
  3. Host clears errors, then calls dlsym(handle, "plugin_get_api").
  4. Host validates api->version and api->size.
  5. Host calls api->init(host_api); plugin returns status.
  6. Host registers plugin commands and stores function pointers.
  7. On unload, host unregisters commands, calls api->shutdown, then dlclose.
  8. Host ensures no thread calls plugin code after unload (lock or refcount).

Minimal Concrete Example

// Required entry symbol in every plugin
plugin_api* plugin_get_api(void);

// Host loader
void *h = dlopen(path, RTLD_NOW | RTLD_LOCAL);
if (!h) return error("dlopen", dlerror());
plugin_api* (*get_api)(void) = dlsym(h, "plugin_get_api");
if (!get_api) return error("dlsym", dlerror());

Common Misconceptions

  • “dlsym validates type safety.” -> It doesn’t; it only returns an address.
  • “If dlopen succeeds, the plugin is safe.” -> It only means the loader found it; ABI may still be incompatible.
  • “Unloading is always safe.” -> It’s unsafe if any stored function pointer can still be called.

Check-Your-Understanding Questions

  1. Why is RTLD_LOCAL safer for plugins than RTLD_GLOBAL?
  2. What happens if a plugin compiled against an older struct layout?
  3. Why should you prefer RTLD_NOW in a plugin system?
  4. What is the failure mode if you dlclose while callbacks remain registered?

Check-Your-Understanding Answers

  1. It prevents symbol collisions and accidental cross-plugin binding.
  2. Field offsets differ; reading fields can access invalid memory.
  3. It fails early if dependencies or symbols are missing.
  4. You may call a dangling function pointer into unmapped memory.

Real-World Applications

  • Image editors loading format plugins at runtime.
  • Databases loading storage engines.
  • CLI tools loading subcommands as plugins.

Where You’ll Apply It

References

  • “The Linux Programming Interface” - Ch. 41-42 (Dynamic Linking)
  • “Linkers and Loaders” by John R. Levine - Ch. 9
  • man dlopen, man dlsym

Key Insight

Dynamic loading turns your process into its own linker; you must enforce safety manually.

Summary

Dynamic loading lets you extend a running process, but it removes compile-time safety. A stable symbol contract, strict validation, and careful unload rules are the difference between a robust plugin system and a crash-prone one.

Homework/Exercises to Practice the Concept

  1. Write a tiny host that loads a plugin and calls a hello() function.
  2. Modify the plugin to change the function signature without changing the header. Observe the crash.
  3. Compare RTLD_NOW vs RTLD_LAZY by creating a plugin with a missing dependency.

Solutions to the Homework/Exercises

  1. Use dlopen/dlsym and call the function pointer; exit cleanly.
  2. The host will call with the wrong ABI; this demonstrates why strict headers matter.
  3. RTLD_NOW fails immediately; RTLD_LAZY fails on first call.

2.2 ABI Contracts, Versioning, and Function Tables

Fundamentals

An ABI (Application Binary Interface) defines how compiled code agrees on data layout, calling conventions, and symbol names. In a plugin system, the ABI is the only thing that matters once the plugin is compiled. A function table (a struct of function pointers) is the most common way to design a stable ABI in C because it lets you add new functions while keeping old fields in place. The ABI must be versioned explicitly, typically with integer version fields and a size field. This lets the host detect older or newer plugins and decide if they are compatible. If you treat ABI stability as optional, every rebuild of a plugin risks breaking users silently.

Deep Dive into the Concept

A stable ABI is about layout discipline. Suppose your plugin_api struct is defined as:

typedef struct {
    uint32_t version;
    uint32_t size;
    int (*init)(const host_api *host);
    int (*register_cmds)(const host_api *host);
    void (*shutdown)(void);
} plugin_api;

If you later add a new function pointer, you must append it to the end and increment the version or size. Old plugins compiled against the earlier layout will have a smaller size, so the host can safely detect missing fields and avoid calling them. This pattern prevents ABI breakage and allows forward evolution. The host must treat size as authoritative: only call functions whose pointers are within size bytes. This is also how many OS ABIs evolve (e.g., Windows structs with cbSize).

Versioning also means defining compatibility rules. A simple policy: host accepts any plugin with version == HOST_API_VERSION and size >= sizeof(min_api_v1). For forward compatibility, you can accept version == 1 but tolerate size > sizeof(v1) and ignore extra fields. For backward compatibility, you can accept version == 0 if you can map or shim older behavior. The key is to document the compatibility matrix and enforce it in code. An implicit policy is a ticking time bomb.

Function tables also create a boundary for ownership. If the host passes a host_api table into init, that table becomes the only “official” way a plugin can call back into the host. This is a control point: you can add new host functionality by adding fields, and you can remove functionality only when you are willing to break ABI. It also encourages dependency inversion: the host defines stable entry points rather than exporting arbitrary global symbols. This reduces accidental ABI leakage.

ABI stability is not just about struct layout. It includes calling conventions (cdecl vs stdcall), integer widths, alignment, and compiler flags. If you build a plugin with different compiler settings (e.g., different -fpack-struct or different default alignment), you can still break ABI even if the struct appears identical. The defensive approach is to mandate build flags in your plugin SDK and provide a single public header that all plugins must include. In practice, you can ship a plugin_api.h plus a pkg-config file that defines required compiler flags.

Naming also matters. If you export plugin_get_api as the entry symbol, you should namespace it or require a standardized name for discovery. While this is technically just API surface, it becomes part of ABI because dlsym relies on the exact string. Avoid changing symbol names after release. If you must, keep the old symbol for compatibility and implement it as a wrapper.

Finally, think about ABI transparency. Provide an api_version constant and a plugin_api_version() function so tools can verify compatibility without loading the plugin. Consider a “manifest” function like plugin_get_metadata() that returns a static struct with plugin name, version, and required host version. This is not strictly necessary but it improves diagnostics and makes your plugin ecosystem more manageable.

How This Fits in This Project

You will define plugin_api and host_api structs and use them as the only cross-boundary contracts. This drives your loader design (Sec. 4.2) and your compatibility checks (Sec. 5.10 Phase 1). The same ABI design strategy is reused in Project 2’s stable client API and Project 5’s public HTTP API. Also used in: Project 2, Project 5.

Definitions & Key Terms

  • ABI -> Binary contract for data layout and calling convention.
  • Function table -> Struct of callbacks implementing an interface.
  • Version field -> Explicit integer indicating API generation.
  • Size field -> Size of struct to detect layout compatibility.
  • Forward compatibility -> New host accepts older plugin.
  • Backward compatibility -> Old host accepts newer plugin.

Mental Model Diagram (ASCII)

host_api (provided by host)        plugin_api (provided by plugin)
+---------------------------+      +-----------------------------+
| version | size            |      | version | size              |
| log()   | register_cmd()  |<---->| init()  | register_cmds()   |
| ...     |                 |      | run()   | shutdown()        |
+---------------------------+      +-----------------------------+

How It Works (Step-by-Step)

  1. Host defines HOST_API_VERSION and struct host_api.
  2. Plugin implements plugin_get_api() returning struct plugin_api.
  3. Host loads plugin and reads api->version and api->size.
  4. Host rejects incompatible versions or too-small sizes.
  5. Host calls api->init(&host_api).
  6. Host only calls function pointers that exist in the plugin’s size range.

Minimal Concrete Example

#define PLUGIN_API_VERSION 1

typedef struct {
    uint32_t version;
    uint32_t size;
    int (*init)(const host_api* host);
    int (*register_cmds)(const host_api* host);
    void (*shutdown)(void);
} plugin_api;

Common Misconceptions

  • “A version number is enough.” -> Size checks are needed to handle layout changes safely.
  • “If I add a field in the middle, old plugins still work.” -> Offsets change; ABI breaks.
  • “Only function signatures matter.” -> Alignment and compiler flags can also break ABI.

Check-Your-Understanding Questions

  1. Why should new fields be appended to a function table?
  2. What does the size field protect against?
  3. How can compiler flags break ABI even if headers match?

Check-Your-Understanding Answers

  1. Appending preserves offsets of existing fields.
  2. It lets the host detect older layouts and avoid reading beyond them.
  3. Flags can change alignment or calling conventions, altering layout.

Real-World Applications

  • OS kernel module interfaces.
  • Browser extension APIs.
  • Database storage engine plugins.

Where You’ll Apply It

  • In this project: Sec. 3.2 (requirements), Sec. 4.2 (components), Sec. 5.11 (decisions).
  • Also used in: Project 2, Project 5.

References

  • “C Interfaces and Implementations” - Ch. 1-2
  • “Expert C Programming” - Ch. 5
  • “Linkers and Loaders” - Ch. 7-9

Key Insight

ABI stability is about struct layout discipline and explicit compatibility rules, not just version numbers.

Summary

Function tables plus version/size fields are the safest way to evolve plugin ABIs. This design keeps old plugins working while letting you add new features over time.

Homework/Exercises to Practice the Concept

  1. Define a v1 plugin_api and a v2 with an added field. Implement host checks.
  2. Build a plugin with a mismatched header and observe detection.
  3. Add a metadata function and print it during plugin listing.

Solutions to the Homework/Exercises

  1. Append the new function pointer and increment size/version.
  2. Host should reject with “incompatible API size/version.”
  3. Use plugin_get_metadata() returning a static struct.

2.3 Plugin Lifecycle, Ownership, and Error Models

Fundamentals

Lifecycle design defines when a plugin is created, initialized, used, and destroyed. Ownership rules specify who allocates and frees plugin state. Error models define how failures are reported and how partial initialization is handled. In a plugin system, lifecycle and ownership are the difference between safe unloads and use-after-free crashes. A clean lifecycle contract states: who calls init, what happens on failure, whether shutdown is always called, and how to unregister commands. Without these rules, plugins can leak resources, leave stale registrations, or corrupt host state.

Deep Dive into the Concept

A plugin is not just a collection of functions; it often carries state (configuration, open files, caches). You must decide where that state lives and who owns it. There are two common patterns. Pattern A: the plugin manages its own internal static state. This is simple but makes it harder to host multiple instances or to unload safely. Pattern B: the plugin returns an opaque state pointer from init and the host passes it into every callback. This is more explicit and safer for multiple instances. For a shell plugin system, Pattern B scales better: you can load two instances of the same plugin with different configurations and manage their lifetimes independently.

Error handling is tightly coupled. If init fails after allocating resources, who cleans them up? A safe contract says: init is responsible for cleaning up on failure, and only returns a valid handle on success. Alternatively, init can return a partially built handle plus an error code, but then the host must call shutdown even on failed init. That contract is harder to use correctly. The simplest rule is: on failure, the plugin cleans up and returns a failure code; the host must not call shutdown if init failed. This must be documented and enforced.

Lifecycle also includes registration. Plugins often call host->register_command() to expose new commands. The host should treat registration as a transactional step: either all commands register and the plugin is considered active, or none do. If registration fails midway, the host should roll back previous registrations and refuse to activate the plugin. This avoids a half-loaded plugin that only partially works. That rollback is part of the boundary contract and should be spelled out in the API documentation.

Error models need consistency. If a plugin function fails, how does it report the reason? Options: return error codes, return NULL plus a plugin_last_error function, or write into a host-provided buffer. For a plugin system, return codes plus a last_error callback is often the simplest. The host can display errors from plugins without guessing. Also decide whether errors are per-plugin or global: per-plugin state is safer and thread-safe.

Unloading is the hardest boundary. When you unload, you must ensure that no threads are executing plugin code and no references remain. If your shell is single-threaded, you can enforce unload only when idle. If multi-threaded, you need reference counting: increment when a command starts, decrement when it finishes; unload only when count reaches zero. You should also invalidate registrations at unload, so that future command dispatch doesn’t use stale pointers.

Finally, consider reentrancy. If a plugin command calls back into the host (e.g., to execute another command), you must avoid deadlocks. If your host holds a global plugin lock while executing a plugin command, and the plugin calls back into the host, you can deadlock. The safer pattern is: resolve function pointers while holding locks, then release locks before calling into plugin code. This separates “registry protection” from “execution.”

How This Fits in This Project

You will define plugin_state ownership in Sec. 3.2 and specify lifecycle rules in Sec. 3.2 and Sec. 3.7. These rules drive the unload design (Sec. 4.1) and the concurrency safeguards in Sec. 7.2. This same ownership discipline is essential in Project 2’s client library and Project 3’s JSON tree management. Also used in: Project 2, Project 3.

Definitions & Key Terms

  • Lifecycle -> Ordered phases: load, init, register, run, unregister, shutdown, unload.
  • Opaque handle -> Pointer to internal state only managed by the plugin.
  • Ownership -> Responsibility for allocation and deallocation.
  • Reentrancy -> Safe to call again before returning.
  • Transactional registration -> Either all commands register or none do.

Mental Model Diagram (ASCII)

load -> init -> register -> active
  |                  |
  | failure          | unload
  v                  v
cleanup          unregister -> shutdown -> dlclose

How It Works (Step-by-Step)

  1. Host loads plugin and obtains plugin_api.
  2. Host calls init(), which allocates plugin state.
  3. Plugin registers commands via host callbacks.
  4. Host marks plugin active only if registration fully succeeds.
  5. On unload, host unregisters commands, waits for active calls, then calls shutdown().
  6. Plugin frees its internal state in shutdown().

Minimal Concrete Example

typedef struct plugin_state plugin_state;
int plugin_init(const host_api *host, plugin_state **out_state);
int plugin_register(const host_api *host, plugin_state *st);
void plugin_shutdown(plugin_state *st);

Common Misconceptions

  • “shutdown should always run.” -> Only if init succeeded under the simplest contract.
  • “Unregistering is optional.” -> It is required to avoid dangling pointers.
  • “Plugins are always single-threaded.” -> Hosts may call commands concurrently.

Check-Your-Understanding Questions

  1. What is the simplest safe rule for init/shutdown on failure?
  2. Why must command registration be transactional?
  3. How do you prevent unloading a plugin while it is executing?

Check-Your-Understanding Answers

  1. If init fails, the plugin cleans up and the host does not call shutdown.
  2. Partial registration leaves dangling commands and inconsistent state.
  3. Use reference counting or only unload when idle.

Real-World Applications

  • Web servers loading modules with initialization hooks.
  • Games loading scripting plugins at runtime.
  • CLI tools loading subcommands dynamically.

Where You’ll Apply It

  • In this project: Sec. 3.2 (requirements), Sec. 5.10 Phase 2 (activation), Sec. 7.1 (pitfalls).
  • Also used in: Project 2, Project 3.

References

  • “Effective C” - Ch. 6 (Ownership)
  • “Designing Interfaces” by Jenifer Tidwell - Chapter on contracts (conceptual)
  • POSIX dlopen documentation

Key Insight

A plugin system is a lifecycle contract first and a dynamic loader second.

Summary

Explicit lifecycle and ownership rules prevent leaks and crashes. The host must enforce these rules because the compiler cannot.

Homework/Exercises to Practice the Concept

  1. Implement a plugin that allocates state and intentionally fails in register.
  2. Add a reference counter and prevent unload while commands run.
  3. Force a plugin to log errors and expose them via plugin_last_error().

Solutions to the Homework/Exercises

  1. Roll back prior registrations and ensure state is freed on failure.
  2. Block unload until the counter reaches zero.
  3. Store the error string in plugin state and return it via a getter.

3. Project Specification

3.1 What You Will Build

A minimal interactive shell named mini-shell that loads external plugins at runtime. Each plugin exports a single entry point that returns a plugin_api function table. The shell can:

  • discover plugins from a directory
  • load/unload plugins safely
  • list active plugins and their metadata
  • dispatch plugin-provided commands

Excluded: job control, full POSIX shell syntax, scripting language, or sandboxing of plugin code.

3.2 Functional Requirements

  1. Plugin Discovery: Scan a plugin directory for .so files.
  2. Load/Unload: Load plugins with dlopen and unload safely.
  3. Compatibility Check: Validate version and size before use.
  4. Command Registration: Plugins register commands via host API.
  5. Command Dispatch: Shell runs built-in and plugin commands.
  6. Error Reporting: All failures return a structured error code and message.
  7. Metadata Listing: List plugin name, version, and API version.

3.3 Non-Functional Requirements

  • Performance: Load a plugin in < 50 ms on a typical machine.
  • Reliability: No crashes when loading incompatible or broken plugins.
  • Usability: Clear error messages and help output.

3.4 Example Usage / Output

mini> load ./plugins/echo.so
Loaded plugin: echo (api v1)
mini> echo hello world
hello world
mini> unload echo
Unloaded plugin: echo

3.5 Data Formats / Schemas / Protocols

Plugin ABI (header excerpt):

#define PLUGIN_API_VERSION 1

typedef struct {
    uint32_t version;
    uint32_t size;
    const char *name;
    const char *plugin_version;
    int (*init)(const struct host_api *host, void **plugin_state);
    int (*register_cmds)(const struct host_api *host, void *plugin_state);
    void (*shutdown)(void *plugin_state);
} plugin_api;

3.6 Edge Cases

  • Plugin missing plugin_get_api symbol.
  • Plugin API version mismatch.
  • init succeeds but register_cmds fails.
  • User unloads plugin while command is running.
  • Two plugins register the same command name.

3.7 Real World Outcome

A user can run a shell, list plugins, load them, run commands, and unload them without crashes. Errors are explicit and actionable.

3.7.1 How to Run (Copy/Paste)

make
./mini-shell --plugins ./plugins

3.7.2 Golden Path Demo (Deterministic)

  • Use plugins echo.so and date.so shipped in ./plugins.
  • Fixed input sequence with deterministic output format.

3.7.3 If CLI: Exact Terminal Transcript

$ ./mini-shell --plugins ./plugins
mini> plugins
[0] builtin (core)
mini> load ./plugins/echo.so
Loaded plugin: echo (api v1)
mini> echo hello boundary
hello boundary
mini> unload echo
Unloaded plugin: echo
mini> exit
$ echo $?
0

Failure demo (missing symbol):

$ ./mini-shell --plugins ./plugins
mini> load ./plugins/bad.so
Error [PLUGIN_ERR_MISSING_SYMBOL]: plugin_get_api not found
mini> exit
$ echo $?
2

Exit Codes:

  • 0 success / normal exit
  • 2 plugin load failure
  • 3 command not found
  • 4 plugin runtime error

4. Solution Architecture

4.1 High-Level Design

+-------------------+    +---------------------+
| mini-shell        |    | plugin.so           |
| - repl            |    | plugin_get_api()    |
| - plugin manager  |<-->| function table      |
| - command registry|    | init/register/run   |
+-------------------+    +---------------------+
         |
         v
   dlopen/dlsym

4.2 Key Components

| Component | Responsibility | Key Decisions | |———–|—————-|—————| | Plugin Manager | Load/unload, validate ABI | Use version+size checks and RTLD_LOCAL | | Command Registry | Map command names to callbacks | Reject duplicates, supports unregister | | REPL | Parse user input and dispatch commands | Simple split by spaces | | Host API Table | Functions exposed to plugins | Stable function table for ABI evolution |

4.3 Data Structures (No Full Code)

typedef struct {
    char name[64];
    uint32_t api_version;
    void *dl_handle;
    plugin_api *api;
    void *state;
    int active_calls;
} plugin_entry;

4.4 Algorithm Overview

Key Algorithm: Safe Load

  1. Open .so with dlopen.
  2. Resolve plugin_get_api and validate version/size.
  3. Call init, then register_cmds.
  4. Insert into registry only if all steps succeed.

Complexity Analysis:

  • Time: O(n) for loading n plugins (per plugin, symbol lookup is O(1)).
  • Space: O(n) for plugin entries and command registry.

5. Implementation Guide

5.1 Development Environment Setup

# Linux or macOS
cc --version
make --version

5.2 Project Structure

mini-shell/
|-- include/
|   |-- host_api.h
|   |-- plugin_api.h
|-- src/
|   |-- main.c
|   |-- plugin_manager.c
|   |-- registry.c
|-- plugins/
|   |-- echo.c
|   `-- bad.c
|-- Makefile
`-- README.md

5.3 The Core Question You’re Answering

“How do you design a runtime interface that stays safe when both sides can evolve independently?”

5.4 Concepts You Must Understand First

Stop and research these before coding:

  1. ABI vs API compatibility (struct layout, calling conventions).
  2. dlopen/dlsym lifecycle and error handling.
  3. Ownership and lifecycle rules for plugin state.
  4. Thread safety and safe unload rules.

5.5 Questions to Guide Your Design

  1. How will you detect incompatible plugins before calling into them?
  2. Where will plugin state live and who frees it?
  3. What happens if registration partially fails?
  4. How will the host prevent unloading while commands run?

5.6 Thinking Exercise

Draw the command registry and plugin state lifecycle. Mark when state is allocated and freed. What happens if a plugin registers two commands and the second fails?

5.7 The Interview Questions They’ll Ask

  1. Why prefer function tables over direct symbol calls?
  2. How do you version an ABI safely?
  3. What goes wrong if you unload a plugin while executing it?
  4. How do you prevent symbol collisions across plugins?

5.8 Hints in Layers

Hint 1: Start with a single entry point

plugin_api* plugin_get_api(void);

Hint 2: Validate version and size

if (api->version != PLUGIN_API_VERSION || api->size < sizeof(plugin_api)) return ERR_INCOMPAT;

Hint 3: Add a registry lock

pthread_mutex_lock(&registry_lock);

5.9 Books That Will Help

| Topic | Book | Chapter | |——-|——|———| | Dynamic loading | “The Linux Programming Interface” | Ch. 41-42 | | ABI design | “Expert C Programming” | Ch. 5 | | Interfaces | “C Interfaces and Implementations” | Ch. 1-2 | | Ownership | “Effective C” | Ch. 6 |

5.10 Implementation Phases

Phase 1: Foundation (2-3 days)

Goals: define ABI headers; build a plugin loader. Tasks:

  1. Write plugin_api.h and host_api.h with version/size.
  2. Implement dlopen/dlsym loader with errors. Checkpoint: load a plugin and call init successfully.

Phase 2: Core Functionality (3-5 days)

Goals: command registry and dispatch. Tasks:

  1. Implement registry add/remove.
  2. Add plugin register/unregister logic.
  3. Build REPL for command execution. Checkpoint: run plugin commands and unload cleanly.

Phase 3: Safety & Edge Cases (2-3 days)

Goals: compatibility checks and safe unload. Tasks:

  1. Add version/size validation and error codes.
  2. Add active call counters and unload guards.
  3. Add failure tests for incompatible plugins. Checkpoint: mismatched plugins fail with explicit errors, no crashes.

5.11 Key Implementation Decisions

| Decision | Options | Recommendation | Rationale | |———-|———|—————-|———–| | ABI evolution | version only vs version+size | version+size | Detects layout changes safely | | Plugin state | internal static vs opaque handle | opaque handle | Supports multiple instances | | Unload policy | allow anytime vs idle only | idle/refcount | Prevents dangling calls |

6. Testing Strategy

6.1 Test Categories

| Category | Purpose | Examples | |———-|———|———-| | Unit Tests | Validate registry functions | add/remove commands | | Integration Tests | Load plugins | load good/bad plugin | | Edge Case Tests | Incompatible ABI | mismatch size/version |

6.2 Critical Test Cases

  1. Missing symbol: plugin lacks plugin_get_api -> load fails with code.
  2. Version mismatch: host rejects older API.
  3. Unload with active call: unload should block or return error.

6.3 Test Data

plugins/echo.so
plugins/bad.so (missing symbol)
plugins/old.so (older API)

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

| Pitfall | Symptom | Solution | |——–|———|———-| | ABI mismatch | Crash on first call | Enforce version/size checks | | Stale pointers | Crash after unload | Unregister before dlclose | | Missing symbol | Load fails silently | Wrap dlerror() clearly |

7.2 Debugging Strategies

  • Use nm -D to verify exported symbols.
  • Print version/size from both host and plugin to compare.
  • Use AddressSanitizer to catch use-after-free on unload.

7.3 Performance Traps

  • Excessive symbol lookups on every call; cache function pointers.

8. Extensions & Challenges

8.1 Beginner Extensions

  • Add help output that lists plugin commands.
  • Add a plugin info command that prints metadata.

8.2 Intermediate Extensions

  • Add hot-reload by unloading and reloading a plugin.
  • Add dependency checks between plugins.

8.3 Advanced Extensions

  • Add a sandboxed plugin process with IPC boundary.
  • Add ABI compatibility shims for old plugins.

9. Real-World Connections

9.1 Industry Applications

  • Web servers: load authentication modules or filters.
  • Games: load scripting modules and mod content.
  • Nginx modules - dynamic modules with strict ABI policies.
  • Apache httpd - module system with lifecycle hooks.

9.3 Interview Relevance

  • Dynamic loading: explain how dlopen works.
  • ABI stability: demonstrate safe evolution strategies.

10. Resources

10.1 Essential Reading

  • “The Linux Programming Interface” - Ch. 41-42 (Dynamic Linking)
  • “Expert C Programming” - Ch. 5 (Linkage and ABI)

10.2 Video Resources

  • “Dynamic Linking in Linux” - conference talk (searchable title)

10.3 Tools & Documentation

  • dlopen man page - runtime loader API.
  • nm/readelf - inspect exported symbols.

11. Self-Assessment Checklist

11.1 Understanding

  • I can explain how dlopen and dlsym work.
  • I can explain why version+size checks prevent ABI breakage.
  • I can describe safe unload rules.

11.2 Implementation

  • All functional requirements are met.
  • Incompatible plugins fail gracefully.
  • No crashes on load/unload cycles.

11.3 Growth

  • I can explain the plugin lifecycle in an interview.
  • I documented lessons learned and edge cases.

12. Submission / Completion Criteria

Minimum Viable Completion:

  • Load and unload one plugin safely.
  • Validate API version and size.
  • Provide clear error messages for missing symbols.

Full Completion:

  • All commands register/unregister cleanly.
  • Plugins list metadata and API version.
  • Tests cover incompatible plugin cases.

Excellence (Going Above & Beyond):

  • Supports hot-reload without process restart.
  • Provides a compatibility shim for old plugins.