Project 9: plug-master (Plugin Architecture)

Build a CLI that discovers external plugins and exposes them as subcommands safely and reliably.

Quick Reference

Attribute Value
Difficulty Level 4 (Expert)
Time Estimate 3-4 weeks
Main Programming Language Go (Alternatives: Rust)
Alternative Programming Languages Rust
Coolness Level Level 4: Hardcore
Business Potential 4: Platform-level tool
Prerequisites Process execution, IPC, versioning
Key Topics plugin discovery, RPC, isolation, versioning

1. Learning Objectives

By completing this project, you will:

  1. Design a plugin discovery mechanism with naming conventions.
  2. Define a stable plugin protocol and version handshake.
  3. Isolate plugins using separate processes and IPC.
  4. Provide safety boundaries and error handling for third-party code.
  5. Build a CLI that dynamically extends its command tree.

2. All Theory Needed (Per-Concept Breakdown)

2.1 Plugin Discovery and Command Registration

Fundamentals

Plugins are external executables that extend a CLI. The host tool must discover them reliably and register them as subcommands. The most common pattern is a naming convention, such as plug-master-<name>, discovered on the PATH or in a plugins directory. Discovery should be deterministic: given the same PATH and plugin dirs, the list should always be the same. This ensures consistent help output and predictable behavior.

Deep Dive into the concept

Discovery begins with defining where plugins live. You can search the PATH for executables that match a prefix, or you can use a dedicated directory like ~/.local/share/plug-master/plugins. The prefix approach is common because it allows plugins to be installed via package managers. However, scanning PATH can be slow and may include irrelevant executables. The dedicated directory is more controlled but less discoverable. Many tools support both.

Once you find candidate executables, you need a way to query their metadata. The standard approach is to run the plugin with a --plugin-info or metadata command that prints JSON describing the plugin name, version, and supported commands. This allows the host to verify compatibility before invoking the plugin for real. Metadata also provides help text for the command tree. The host can then register each plugin as a subcommand under the main CLI.

Determinism matters here. If two plugins have the same name, you need a conflict rule. Options include: pick the first on PATH, prefer plugins in the dedicated directory, or error with a conflict message. Document the rule and make it stable. It is better to fail than to run an unexpected plugin.

Discovery also intersects with security. Running arbitrary executables can be risky. The host should avoid executing plugins during discovery unless necessary. If it does, it should do so with a timeout and minimal environment variables. The plugin protocol should include a version handshake to prevent incompatible plugins from being loaded.

Finally, command registration must be integrated with help output and completion. The CLI should include plugins in --help output and generate completion scripts that include plugin commands. This is part of the UX contract.

How this fit on projects

This concept defines how plug-master finds plugins and integrates them into the command tree.

Definitions & key terms

  • Plugin discovery: Finding plugin executables.
  • Naming convention: Prefix like plug-master-.
  • Metadata handshake: Plugin reports name and version.
  • Command registration: Adding plugin commands to CLI help.

Mental model diagram (ASCII)

PATH + plugin dir -> scan -> metadata -> register commands

How it works (step-by-step)

  1. Build a list of plugin directories and PATH entries.
  2. Scan for executables matching prefix.
  3. Run each plugin with --plugin-info.
  4. Validate metadata and register command.
  5. Expose plugin commands in help output.

Minimal concrete example

$ plug-master --help
Available plugins:
  plug-master-hello
  plug-master-lint

Common misconceptions

  • “Plugin discovery can be random.” -> It must be deterministic.
  • “Conflicts can be ignored.” -> They create security risks.
  • “Help output does not need plugins.” -> It should include them.

Check-your-understanding questions

  1. Why should plugin discovery be deterministic?
  2. How can you avoid loading incompatible plugins?
  3. What should happen if two plugins share the same name?

Check-your-understanding answers

  1. To ensure stable behavior and help output.
  2. Use a metadata handshake with version checks.
  3. Fail with a conflict error or use a documented priority rule.

Real-world applications

  • kubectl plugins are discovered by naming convention.
  • git uses git-<cmd> executables for extensions.

Where you will apply it

References

  • Kubernetes plugin design docs
  • HashiCorp plugin architecture

Key insights

Discovery is part of the user contract; it must be predictable and safe.

Summary

Plugins are only useful when discovery is consistent, safe, and integrated into the CLI experience.

Homework/Exercises to practice the concept

  1. Design a plugin naming convention.
  2. Implement a scanner that finds executables with that prefix.
  3. Handle naming conflicts deterministically.

Solutions to the homework/exercises

  1. Use plug-master-<name>.
  2. Scan PATH and plugin directory.
  3. Error on duplicates or prefer plugin dir.

2.2 Isolation, IPC, and Protocol Design

Fundamentals

Plugins should run in separate processes to isolate crashes and prevent memory corruption. Inter-process communication (IPC) allows the host and plugin to exchange data. A simple protocol can be JSON over stdin/stdout. The protocol must include versioning so the host can reject incompatible plugins. Isolation also allows plugins to be untrusted without bringing down the host.

Deep Dive into the concept

In-process plugins (shared libraries) can be fast but dangerous. A crash or memory bug in a plugin can bring down the host. Running plugins as separate processes avoids this. The host can spawn a plugin process, send a request, and read the response. If the plugin crashes, the host can report an error and continue.

IPC can be done via stdin/stdout, sockets, or gRPC. For a CLI, JSON over stdin/stdout is simple and portable. Each request can be a JSON object with a command, args, and env. The plugin responds with a JSON object containing stdout, stderr, and exit_code. This makes the plugin behave like a subprocess while still using a structured protocol.

Versioning is crucial. The protocol should have a version field (e.g., api_version: 1). The host should reject plugins that do not match or that declare unsupported versions. This prevents subtle breakages when the host evolves. You should also consider compatibility ranges, such as allowing 1.x versions.

Timeouts and resource limits are part of isolation. The host should enforce a timeout for plugin responses and kill plugins that hang. It can also set environment variables to limit plugin behavior. If a plugin is slow, it should not block the host indefinitely.

Finally, error handling must be strict. If a plugin returns invalid JSON, the host should treat it as a failure and not expose partial output. This protects the host from malformed or malicious plugins.

How this fit on projects

This concept defines the IPC protocol and how plugins communicate with the host. It directly affects reliability and security.

Definitions & key terms

  • IPC: Inter-process communication.
  • Protocol version: Version of the host-plugin API.
  • Isolation: Running plugins in separate processes.
  • Timeout: Maximum allowed execution time.

Mental model diagram (ASCII)

Host -> JSON request -> Plugin process -> JSON response -> Host

How it works (step-by-step)

  1. Host spawns plugin process.
  2. Host sends a JSON request over stdin.
  3. Plugin executes and returns JSON response.
  4. Host validates response and renders output.
  5. Host enforces timeouts and exits gracefully.

Minimal concrete example

{"api_version":1,"command":"hello","args":["--name","world"]}

Common misconceptions

  • “In-process plugins are faster and safe.” -> They are unsafe by default.
  • “Protocol versioning is optional.” -> It prevents breaking changes.
  • “Timeouts are only for network code.” -> Plugins can hang too.

Check-your-understanding questions

  1. Why run plugins as separate processes?
  2. How does protocol versioning protect the host?
  3. Why should the host validate plugin responses?

Check-your-understanding answers

  1. It isolates crashes and memory errors.
  2. It prevents incompatible plugins from loading.
  3. To avoid malformed data and security issues.

Real-world applications

  • HashiCorp uses gRPC-based plugin protocols.
  • kubectl plugins are separate executables.

Where you will apply it

References

  • HashiCorp go-plugin docs
  • gRPC and JSON-RPC references

Key insights

Process isolation and a versioned protocol are the foundation of safe plugin systems.

Summary

Use separate processes, a structured protocol, and strict validation to keep plugins safe.

Homework/Exercises to practice the concept

  1. Design a JSON request/response format for a plugin.
  2. Implement a plugin that responds to a ping command.
  3. Add version checks in the host.

Solutions to the homework/exercises

  1. Include api_version, command, args fields.
  2. Return {"status":"ok"} for ping.
  3. Reject if api_version mismatch.

2.3 Security Boundaries and Capability Control

Fundamentals

Plugins are third-party code. Even if they are trusted, they should not be allowed to do anything without limits. Capability control defines what plugins can access: file system, network, environment variables. At minimum, you should restrict what environment variables are passed to plugins and provide a way to disable plugins entirely. Security boundaries are what allow a plugin ecosystem to exist without compromising the host.

Deep Dive into the concept

The host process can enforce security boundaries in several ways. It can run plugins with a restricted environment, removing sensitive variables. It can set a working directory or restrict file access by convention. For stronger isolation, it can run plugins in containers or with OS sandboxing (e.g., macOS sandbox, Linux namespaces). For this project, a lighter-weight boundary is enough: sanitize the environment and disallow plugins from modifying global config unless explicitly allowed.

Capability control also includes an allowlist of plugin commands. The host should define which plugin commands are allowed to run, and it should not automatically execute plugins on startup unless necessary. Provide a plugin enable and plugin disable mechanism so users can manage plugins explicitly. This is important for security and stability.

Logging is another boundary. If plugins can print arbitrary output, they may confuse users or attempt to spoof host messages. Use a consistent prefix or formatting to distinguish plugin output. Also, capture plugin stderr separately and show it in a controlled way.

Finally, versioning and signing can provide trust. A plugin can include a signature, and the host can verify it against a known key. This is an advanced extension but worth mentioning. Even without signatures, you should document that plugins are equivalent to running executables and should be installed from trusted sources.

How this fit on projects

This concept defines plugin safety mechanisms, environment sanitization, and management commands.

Definitions & key terms

  • Capability: Allowed action for a plugin.
  • Sandbox: Environment with restricted access.
  • Allowlist: Explicit list of enabled plugins.
  • Signature: Cryptographic proof of authenticity.

Mental model diagram (ASCII)

Host env -> sanitize -> plugin env
Host -> allowlist -> enabled plugins only

How it works (step-by-step)

  1. Load enabled plugin list.
  2. Sanitize environment variables.
  3. Spawn plugin with restricted context.
  4. Capture output and prefix it.
  5. Provide enable/disable commands.

Minimal concrete example

$ plug-master plugin disable plug-master-unsafe

Common misconceptions

  • “Plugins are trusted by default.” -> Treat them as untrusted.
  • “Env vars are harmless.” -> They can leak secrets.
  • “Disable is unnecessary.” -> Users need control.

Check-your-understanding questions

  1. Why should plugin env vars be sanitized?
  2. What is the purpose of an allowlist?
  3. How can plugin output be made safe for users?

Check-your-understanding answers

  1. To prevent leaking sensitive information.
  2. To ensure only trusted plugins are active.
  3. Prefix plugin output and keep stderr separate.

Real-world applications

  • Browser extensions use permission systems.
  • CLI plugin managers allow enable/disable lists.

Where you will apply it

References

  • Software architecture texts on plugin systems
  • OWASP guidelines on untrusted code

Key insights

A plugin system without capability boundaries is a liability, not a feature.

Summary

Security boundaries and explicit enablement keep plugin ecosystems safe and controllable.

Homework/Exercises to practice the concept

  1. Define a minimal sanitized env for plugins.
  2. Implement a plugin allowlist file.
  3. Add output prefixing for plugin messages.

Solutions to the homework/exercises

  1. Pass only PATH, HOME, and explicit allowlist vars.
  2. Store enabled plugins in config.
  3. Prefix output lines with plugin:.

3. Project Specification

3.1 What You Will Build

A CLI named plug-master that discovers plugins, registers them as subcommands, and runs them via a structured protocol. It supports plugin enable/disable, version checks, and safe execution. It exposes plugin help and includes them in completion scripts.

3.2 Functional Requirements

  1. Discovery: scan PATH and plugin dir for plug-master-*.
  2. Metadata handshake: --plugin-info JSON with version.
  3. Protocol: JSON request/response with versioning.
  4. Isolation: run plugins as separate processes.
  5. Management: enable/disable plugins.
  6. Help integration: include plugins in help output.

3.3 Non-Functional Requirements

  • Security: sanitized environment and strict validation.
  • Reliability: timeouts and crash isolation.
  • Usability: clear error messages for plugin failures.

3.4 Example Usage / Output

$ plug-master --help
Available plugins:
  plug-master-hello
  plug-master-lint

$ plug-master hello --name world
Hello, world!

3.5 Data Formats / Schemas / Protocols

{
  "api_version": 1,
  "command": "hello",
  "args": ["--name", "world"]
}

3.6 Edge Cases

  • Plugin metadata missing -> ignore plugin with warning.
  • Version mismatch -> exit code 3.
  • Plugin timeout -> exit code 4.

3.7 Real World Outcome

3.7.1 How to Run (Copy/Paste)

# Build
go build -o plug-master ./cmd/plug-master

# Run
./plug-master --help

3.7.2 Golden Path Demo (Deterministic)

$ ./plug-master hello --name world
Hello, world!
$ echo $?
0

3.7.3 Failure Demo (Deterministic)

$ ./plug-master broken
plug-master: plugin not found: broken
$ echo $?
2

3.7.4 Exit Codes

  • 0: Success.
  • 2: Plugin not found.
  • 3: Version mismatch.
  • 4: Plugin timeout or crash.

4. Solution Architecture

4.1 High-Level Design

+------------------+
| Discovery Engine |
+------------------+
          |
          v
+------------------+     +------------------+
| Plugin Registry  | --> | Protocol Handler |
+------------------+     +------------------+
          |                        |
          v                        v
+------------------+     +------------------+
| Command Router   | --> | Process Runner   |
+------------------+     +------------------+

4.2 Key Components

Component Responsibility Key Decisions
Discovery find plugins prefix + dir scan
Registry store metadata version checks
Router map commands dynamic subcommands
Runner spawn plugin timeouts and isolation

4.3 Data Structures (No Full Code)

type PluginInfo struct {
    Name string
    Version string
    ApiVersion int
}

4.4 Algorithm Overview

Key Algorithm: Plugin Execution

  1. Resolve plugin name to executable.
  2. Validate metadata and version.
  3. Spawn process and send JSON request.
  4. Read JSON response and render output.

Complexity Analysis:

  • Time: O(P) for discovery.
  • Space: O(P) for registry.

5. Implementation Guide

5.1 Development Environment Setup

mkdir plug-master && cd plug-master

5.2 Project Structure

cmd/plug-master/
  main.go
internal/
  discover/
  protocol/
  registry/
  run/

5.3 The Core Question You’re Answering

“How do I let third parties extend my CLI without breaking it?”

5.4 Concepts You Must Understand First

  1. Plugin discovery and metadata.
  2. IPC protocol and versioning.
  3. Security boundaries and capability control.

5.5 Questions to Guide Your Design

  1. Where should plugins be stored?
  2. How do you avoid running incompatible plugins?
  3. How do you handle plugin crashes?

5.6 Thinking Exercise

Design a handshake protocol that includes plugin name, version, and API version.

5.7 The Interview Questions They’ll Ask

  1. “How do you discover plugins safely?”
  2. “Why run plugins in separate processes?”
  3. “How do you version plugin APIs?”

5.8 Hints in Layers

Hint 1: Naming convention Use plug-master-<name>.

Hint 2: Metadata command Require --plugin-info to return JSON.

Hint 3: Version handshake Reject mismatched API versions.

Hint 4: Timeouts Kill plugins that hang.

5.9 Books That Will Help

Topic Book Chapter
Modularity Fundamentals of Software Architecture Ch. 6
Processes Advanced Programming in the UNIX Environment process chapters

5.10 Implementation Phases

Phase 1: Discovery (1 week)

Goals: scan and list plugins.

Phase 2: Protocol (1 week)

Goals: request/response and versioning.

Phase 3: Safety and Management (1 week)

Goals: enable/disable, timeouts, logging.

5.11 Key Implementation Decisions

Decision Options Recommendation Rationale
Discovery PATH vs plugin dir both flexibility.
IPC JSON vs gRPC JSON simplicity.
Isolation process vs in-process process safety.

6. Testing Strategy

6.1 Test Categories

Category Purpose Examples
Unit Tests metadata parsing JSON schema validation
Integration Tests plugin execution hello plugin run
Edge Case Tests version mismatch exit code 3

6.2 Critical Test Cases

  1. Plugin discovered and registered correctly.
  2. Version mismatch rejects plugin.
  3. Plugin timeout is enforced.

6.3 Test Data

fixtures/plugins/plug-master-hello

7. Common Pitfalls and Debugging

7.1 Frequent Mistakes

Pitfall Symptom Solution
Loading incompatible plugins crashes enforce version checks
Running plugins on discovery slow startup cache metadata
Untrusted plugin output confusing logs prefix output

7.2 Debugging Strategies

  • Add --debug to show plugin discovery paths.
  • Provide a plugin doctor command.
  • Use timeouts and log if a plugin exceeds them.

7.3 Performance Traps

  • Scanning PATH on every command can be slow; cache results.

8. Extensions and Challenges

8.1 Beginner Extensions

  • Add plugin list command.
  • Add plugin info <name>.

8.2 Intermediate Extensions

  • Add plugin version constraints.
  • Add plugin install command.

8.3 Advanced Extensions

  • Add plugin signing and verification.
  • Add sandboxing with containers.

9. Real-World Connections

9.1 Industry Applications

  • Developer platforms that allow third-party CLI extensions.
  • HashiCorp plugins, kubectl plugins.

9.3 Interview Relevance

  • Plugin architecture and process isolation are common system design topics.

10. Resources

10.1 Essential Reading

  • Fundamentals of Software Architecture, Ch. 6
  • HashiCorp plugin docs

10.2 Video Resources

  • “Designing Plugin Systems” talks

10.3 Tools and Documentation

  • gRPC or JSON-RPC docs

11. Self-Assessment Checklist

11.1 Understanding

  • I can explain plugin discovery and naming conventions.
  • I can explain IPC protocol versioning.
  • I can explain why isolation matters.

11.2 Implementation

  • Plugins are discovered deterministically.
  • Version mismatches are rejected.
  • Plugin failures do not crash the host.

11.3 Growth

  • I can add a new plugin safely.
  • I can debug discovery problems.
  • I can explain security boundaries.

12. Submission / Completion Criteria

Minimum Viable Completion:

  • Plugin discovery and help integration.
  • Versioned protocol and execution.

Full Completion:

  • Enable/disable management and timeouts.

Excellence (Going Above and Beyond):

  • Plugin signing and sandboxing.