Stream Deck Plugin Development Learning Projects
Goal: Master the complete Stream Deck plugin development lifecycle—from understanding how hardware buttons communicate with software through WebSockets, to building production-quality plugins with dynamic UIs, persistent settings, and real-time visual feedback on physical LCD keys.
By completing these projects, you will understand:
- How desktop applications communicate with hardware peripherals via WebSockets
- Event-driven programming patterns in a tangible, visual context
- Building configuration UIs (Property Inspectors) that persist user preferences
- Dynamic image generation and real-time rendering on constrained displays
- Plugin architecture patterns applicable to VS Code extensions, Electron apps, and browser extensions
Why Stream Deck Plugin Development Matters
The Stream Deck isn’t just a macro keypad—it’s a window into how modern desktop applications integrate with hardware. Every skill you learn here transfers directly:
Stream Deck Plugin Development → Transferable Skills
↓ ↓
WebSocket Communication → Real-time apps, chat systems, live data
Event-Driven Architecture → Node.js, Electron, browser extensions
Manifest-Based Configuration → VS Code extensions, Chrome extensions
Property Inspector UIs → Embedded web views, Electron panels
Dynamic Image Rendering → Canvas APIs, data visualization
Plugin Lifecycle Management → Any plugin/extension ecosystem
The Industry Context
Stream Deck plugins represent a microcosm of modern application development:
| Industry Pattern | Stream Deck Implementation |
|---|---|
| Microservices communication | Plugin ↔ Stream Deck app via WebSocket |
| Configuration as code | manifest.json defining all plugin behavior |
| Event sourcing | Actions triggered by hardware events |
| Real-time updates | Dynamic key images reflecting system state |
| Settings persistence | Global and per-action settings storage |
Understanding these patterns deeply will make you a better developer regardless of your primary tech stack.
Stream Deck Architecture Deep Dive
The Big Picture: How It All Connects
When you press a button on your Stream Deck, here’s what actually happens:
┌─────────────────────────────────────────────────────────────────────────────┐
│ YOUR PHYSICAL STREAM DECK │
│ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ │
│ │ LCD │ │ LCD │ │ LCD │ │ LCD │ │ LCD │ │ LCD │ │ LCD │ │ LCD │ ← Keys │
│ │ Key │ │ Key │ │ Key │ │ Key │ │ Key │ │ Key │ │ Key │ │ Key │ │
│ └──┬──┘ └──┬──┘ └──┬──┘ └──┬──┘ └──┬──┘ └──┬──┘ └──┬──┘ └──┬──┘ │
│ │ │ │ │ │ │ │ │ │
│ └───────┴───────┴───────┴───────┼───────┴───────┴───────┘ │
│ │ │
│ USB Connection │
└─────────────────────────────────────┼───────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ STREAM DECK APPLICATION │
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Hardware Communication Layer │ │
│ │ Receives: Button presses, dial rotations, touch events │ │
│ │ Sends: Image data, LED colors, display updates │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Plugin Manager │ │
│ │ • Spawns one process per plugin at startup │ │
│ │ • Establishes WebSocket connection with each plugin │ │
│ │ • Routes events to appropriate plugins │ │
│ │ • Manages plugin lifecycle (start, stop, reload) │ │
│ └──────────────────────────┬──────────────────────────────────────────┘ │
│ │ │
│ ┌───────────────┼───────────────┐ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ WebSocket:9001 WebSocket:9002 WebSocket:9003 │
│ │ │ │ │
└──────────────┼───────────────┼───────────────┼──────────────────────────────┘
│ │ │
▼ ▼ ▼
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ YOUR PLUGIN │ │ Other Plugin │ │ Other Plugin │
│ (Node.js) │ │ (Node.js) │ │ (Python/C++) │
│ │ │ │ │ │
│ ┌──────────────┐ │ │ ┌──────────────┐ │ │ ┌──────────────┐ │
│ │ Backend │ │ │ │ Backend │ │ │ │ Backend │ │
│ │ (plugin.ts) │ │ │ │ (plugin.ts) │ │ │ │ (main.py) │ │
│ └──────────────┘ │ │ └──────────────┘ │ │ └──────────────┘ │
│ │ │ │ │ │
│ ┌──────────────┐ │ │ ┌──────────────┐ │ │ ┌──────────────┐ │
│ │ Property │ │ │ │ Property │ │ │ │ Property │ │
│ │ Inspector │ │ │ │ Inspector │ │ │ │ Inspector │ │
│ │ (HTML/JS) │ │ │ │ (HTML/JS) │ │ │ │ (HTML/JS) │ │
│ └──────────────┘ │ │ └──────────────┘ │ │ └──────────────┘ │
└──────────────────┘ └──────────────────┘ └──────────────────┘
Key Insight: The Dual-Process Model
Every Stream Deck plugin runs as two separate processes that communicate independently with the Stream Deck application:
┌─────────────────────────────────────────────────────────────────────────┐
│ YOUR PLUGIN BUNDLE │
│ (com.yourname.yourplugin.sdPlugin) │
│ │
│ ┌─────────────────────────────────┐ ┌───────────────────────────┐ │
│ │ BACKEND │ │ PROPERTY INSPECTOR │ │
│ │ (plugin.ts) │ │ (UI HTML) │ │
│ │ │ │ │ │
│ │ • Long-running Node.js process │ │ • Web view (like iframe) │ │
│ │ • Handles all action logic │ │ • Opens when user clicks │ │
│ │ • Manages state & timers │ │ action in Stream Deck │ │
│ │ • Calls external APIs │ │ • Renders settings form │ │
│ │ • Updates key images │ │ • User-facing config UI │ │
│ │ │ │ │ │
│ │ Events received: │ │ Events received: │ │
│ │ • onKeyDown / onKeyUp │ │ • onDidReceiveSettings │ │
│ │ • onWillAppear │ │ • onSendToPI │ │
│ │ • onDialRotate │ │ │ │
│ │ • onDidReceiveSettings │ │ Commands sent: │ │
│ │ │ │ • setSettings │ │
│ │ Commands sent: │ │ • sendToPlugin │ │
│ │ • setImage │ │ │ │
│ │ • setTitle │ │ │ │
│ │ • setSettings │ │ │ │
│ │ • sendToPropertyInspector │ │ │ │
│ └───────────────┬─────────────────┘ └─────────────┬─────────────┘ │
│ │ │ │
│ │ WebSocket │ WebSocket │
│ │ Connection │ Connection │
│ │ │ │
└───────────────────┼────────────────────────────────────┼─────────────────┘
│ │
└──────────────┬─────────────────────┘
│
▼
┌──────────────────────────────────┐
│ STREAM DECK APPLICATION │
│ (Routes messages between all │
│ parties and the hardware) │
└──────────────────────────────────┘
The WebSocket Message Flow
All communication uses JSON messages over WebSocket. Here’s what a typical interaction looks like:
User presses Stream Deck App Your Plugin Backend
a key routes event receives & handles
│ │ │
│ ──── USB Signal ────────► │ │
│ │ │
│ │ ──── WebSocket JSON ─────► │
│ │ │
│ │ { │
│ │ "event": "keyDown", │
│ │ "action": "com.you.timer",
│ │ "context": "abc123",│
│ │ "payload": { │
│ │ "settings": {...},│
│ │ "coordinates": { │
│ │ "row": 0, │
│ │ "column": 2 │
│ │ } │
│ │ } │
│ │ } │
│ │ │
│ │ ◄──── setImage command ─── │
│ │ │
│ ◄─── Update LCD ───────── │ { │
│ Display │ "event": "setImage",│
│ │ "context": "abc123",│
│ │ "payload": { │
│ │ "image": "data:..."│
│ │ } │
│ │ } │
The Manifest: Your Plugin’s Blueprint
The manifest.json file is the single source of truth for your plugin. Stream Deck reads this file to understand what your plugin does, what actions it provides, and how to display it.
┌─────────────────────────────────────────────────────────────────────────────┐
│ manifest.json │
│ │
│ ┌────────────────────────────────────────────────────────────────────────┐ │
│ │ PLUGIN METADATA │ │
│ │ │ │
│ │ Name: "My Awesome Plugin" │ │
│ │ Version: "1.0.0" │ │
│ │ Author: "Your Name" │ │
│ │ Description: "Does awesome things" │ │
│ │ Category: "Custom" │ │
│ │ Icon: "plugin-icon.png" │ │
│ └────────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────────────────────────┐ │
│ │ ENTRY POINTS │ │
│ │ │ │
│ │ CodePath: "bin/plugin.js" ← Your compiled TypeScript │ │
│ │ CodePathWin: "bin/plugin.exe" ← Optional Windows-specific │ │
│ │ CodePathMac: "bin/plugin" ← Optional macOS-specific │ │
│ └────────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────────────────────────┐ │
│ │ ACTIONS │ │
│ │ ┌──────────────────────────────────────────────────────────────────┐ │ │
│ │ │ Action 1: "Start Timer" │ │ │
│ │ │ UUID: "com.yourname.plugin.start-timer" │ │ │
│ │ │ Icon: "actions/timer/icon.png" │ │ │
│ │ │ PropertyInspectorPath: "ui/timer-settings.html" │ │ │
│ │ │ States: [ { Image: "timer-off.png" }, { Image: "timer-on" } ] │ │ │
│ │ │ SupportedControllers: ["Keypad", "Encoder"] │ │ │
│ │ └──────────────────────────────────────────────────────────────────┘ │ │
│ │ ┌──────────────────────────────────────────────────────────────────┐ │ │
│ │ │ Action 2: "Reset Timer" │ │ │
│ │ │ UUID: "com.yourname.plugin.reset-timer" │ │ │
│ │ │ Icon: "actions/reset/icon.png" │ │ │
│ │ │ ... │ │ │
│ │ └──────────────────────────────────────────────────────────────────┘ │ │
│ └────────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────────────────────────┐ │
│ │ REQUIREMENTS │ │
│ │ │ │
│ │ Software.MinimumVersion: "6.9" │ │
│ │ Nodejs.Version: "20" │ │
│ │ OS: [{ Platform: "mac" }, { Platform: "windows" }] │ │
│ └────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
Critical Manifest Fields Explained
{
// IDENTITY - How Stream Deck identifies your plugin
"UUID": "com.yourname.yourplugin", // MUST be unique worldwide
"Name": "My Plugin", // Display name in Stream Deck store
"Version": "1.0.0", // Semantic versioning
// ENTRY POINT - Where your code lives
"CodePath": "bin/plugin.js", // Main backend script
// ACTIONS - Each button type your plugin provides
"Actions": [
{
"UUID": "com.yourname.yourplugin.action1", // Unique action ID
"Name": "Do Something", // Button name in UI
"Icon": "actions/action1/icon", // Icon path (no extension)
"Tooltip": "Does something cool", // Hover tooltip
// STATES - Different visual states for the button
"States": [
{ "Image": "actions/action1/off" }, // State 0 (default)
{ "Image": "actions/action1/on" } // State 1 (active)
],
// PROPERTY INSPECTOR - Settings UI for this action
"PropertyInspectorPath": "ui/action1.html",
// SUPPORTED DEVICES
"SupportedControllers": ["Keypad", "Encoder"] // Keys, dials, or both
}
]
}
Actions & Events: The Core Interaction Model
Actions are the heart of Stream Deck development. Each action is a class that responds to events:
┌─────────────────────────────────────────────────────────────────────────────┐
│ ACTION LIFECYCLE EVENTS │
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ APPEARANCE EVENTS │ │
│ │ │ │
│ │ onWillAppear ─────────► Action becomes visible on a key │ │
│ │ (user adds it to their profile) │ │
│ │ │ │
│ │ onWillDisappear ──────► Action is removed from visibility │ │
│ │ (user removes or switches profile) │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ INTERACTION EVENTS │ │
│ │ │ │
│ │ onKeyDown ────────────► User presses a key (finger down) │ │
│ │ onKeyUp ──────────────► User releases a key (finger up) │ │
│ │ │ │
│ │ onDialRotate ─────────► User rotates encoder dial (Stream Deck +) │ │
│ │ onDialPress ──────────► User presses encoder dial │ │
│ │ │ │
│ │ onTouchTap ───────────► User taps touch screen (Stream Deck +) │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ SETTINGS EVENTS │ │
│ │ │ │
│ │ onDidReceiveSettings ──► Settings were updated (from PI or API) │ │
│ │ │ │
│ │ onSendToPlugin ────────► Property Inspector sent a message │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ SYSTEM EVENTS │ │
│ │ │ │
│ │ onSystemDidWakeUp ────► Computer woke from sleep │ │
│ │ │ │
│ │ onApplicationDidLaunch ► Monitored app launched │ │
│ │ onApplicationDidTerminate ► Monitored app closed │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
Event Flow Example: Button Press
TIME ───────────────────────────────────────────────►
User adds action User presses User releases User removes
to Stream Deck the key the key the action
│ │ │ │
▼ ▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐
│onWill │ │onKey │ │onKey │ │onWill │
│Appear │ │Down │ │Up │ │Disappear│
└────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘
│ │ │ │
▼ ▼ ▼ ▼
Initialize Start your Complete your Cleanup
your action action logic action logic resources
state (start timer, (save data, (stop timers,
toggle state) update UI) close connections)
Action Implementation Pattern
// actions/timer.ts
import { action, SingletonAction, KeyDownEvent, WillAppearEvent } from "@elgato/streamdeck";
@action({ UUID: "com.yourname.plugin.timer" })
export class TimerAction extends SingletonAction {
// Called once when action appears on any key
override async onWillAppear(ev: WillAppearEvent): Promise<void> {
// Initialize state from saved settings
const settings = ev.payload.settings;
this.initializeTimer(settings);
// Set initial image
await ev.action.setImage("data:image/png;base64,...");
}
// Called every time user presses the key
override async onKeyDown(ev: KeyDownEvent): Promise<void> {
if (this.isRunning) {
this.pauseTimer();
await ev.action.setState(0); // Show "paused" state
} else {
this.startTimer();
await ev.action.setState(1); // Show "running" state
}
}
// Called when settings change (from Property Inspector)
override async onDidReceiveSettings(ev: DidReceiveSettingsEvent): Promise<void> {
const { duration, breakTime } = ev.payload.settings;
this.updateConfiguration(duration, breakTime);
}
}
Settings Management: Persistence That Works
Stream Deck provides two levels of settings:
┌─────────────────────────────────────────────────────────────────────────────┐
│ SETTINGS HIERARCHY │
│ │
│ ┌────────────────────────────────────────────────────────────────────────┐ │
│ │ GLOBAL SETTINGS │ │
│ │ │ │
│ │ Scope: Entire plugin (shared across all action instances) │ │
│ │ Use for: API tokens, user preferences, feature flags │ │
│ │ │ │
│ │ Example: │ │
│ │ { │ │
│ │ "homeAssistantToken": "eyJhbGciOiJIUzI1NiI...", │ │
│ │ "homeAssistantUrl": "http://192.168.1.100:8123", │ │
│ │ "theme": "dark" │ │
│ │ } │ │
│ │ │ │
│ │ Access: streamDeck.settings.getGlobalSettings() │ │
│ │ Save: streamDeck.settings.setGlobalSettings(settings) │ │
│ └────────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────────────────────────┐ │
│ │ ACTION SETTINGS │ │
│ │ │ │
│ │ Scope: Single action instance (each key has its own settings) │ │
│ │ Use for: Per-button configuration │ │
│ │ │ │
│ │ Example (Timer on Key 1): Example (Timer on Key 5): │ │
│ │ { { │ │
│ │ "workDuration": 25, "workDuration": 50, │ │
│ │ "breakDuration": 5, "breakDuration": 10, │ │
│ │ "soundEnabled": true "soundEnabled": false │ │
│ │ } } │ │
│ │ │ │
│ │ Access: ev.payload.settings (in event handlers) │ │
│ │ Save: ev.action.setSettings(settings) │ │
│ └────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
Settings Flow Between Plugin and Property Inspector
┌──────────────────┐ ┌──────────────────┐
│ │ │ │
│ BACKEND │ │ PROPERTY │
│ (plugin.ts) │ │ INSPECTOR │
│ │ │ (settings.html) │
│ │ │ │
│ ┌──────────┐ │ │ ┌──────────┐ │
│ │ Settings │ │ │ │ Form UI │ │
│ │ Object │ │ │ │ │ │
│ └────┬─────┘ │ │ └────┬─────┘ │
│ │ │ │ │ │
│ │ │ ◄── getSettings() ──── │ │ │
│ │ │ │ User opens │
│ │ │ ── Settings JSON ──────► │ the PI │
│ │ │ │ │ │
│ │ │ │ ▼ │
│ │ │ │ Form displays │
│ │ │ │ current values │
│ │ │ │ │ │
│ │ │ │ User changes │
│ │ │ │ a value │
│ │ │ │ │ │
│ │ │ ◄── setSettings({...}) ── │ │ │
│ │ │ │ │ │
│ ▼ │ │ │
│ onDidReceive │ │ │
│ Settings() │ │ │
│ called │ │ │
│ │ │ │
└──────────────────┘ └──────────────────┘
The Property Inspector: Building Configuration UIs
The Property Inspector is an HTML/CSS/JS web view that appears when users configure an action:
┌─────────────────────────────────────────────────────────────────────────────┐
│ STREAM DECK APPLICATION │
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ MAIN INTERFACE │ │
│ │ ┌─────────────────────────────────────────────────────────────┐ │ │
│ │ │ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ │ │ │
│ │ │ │Timer│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
│ │ │ │ 25 │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
│ │ │ └─────┘ └─────┘ └─────┘ └─────┘ └─────┘ └─────┘ └─────┘ │ │ │
│ │ │ ▲ │ │ │
│ │ │ │ │ │ │
│ │ │ User clicks │ │ │
│ │ │ to configure │ │ │
│ │ └─────────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌─────────────────────────────────────────────────────────────┐ │ │
│ │ │ PROPERTY INSPECTOR (YOUR HTML) │ │ │
│ │ │ ┌───────────────────────────────────────────────────────┐ │ │ │
│ │ │ │ │ │ │ │
│ │ │ │ Timer Settings │ │ │ │
│ │ │ │ ───────────────── │ │ │ │
│ │ │ │ │ │ │ │
│ │ │ │ Work Duration: [ 25 ▼] minutes │ │ │ │
│ │ │ │ │ │ │ │
│ │ │ │ Break Duration: [ 5 ▼] minutes │ │ │ │
│ │ │ │ │ │ │ │
│ │ │ │ ☑ Play sound when timer ends │ │ │ │
│ │ │ │ │ │ │ │
│ │ │ │ Sound: [ Chime ▼] │ │ │ │
│ │ │ │ │ │ │ │
│ │ │ └───────────────────────────────────────────────────────┘ │ │ │
│ │ └─────────────────────────────────────────────────────────────┘ │ │
│ └──────────────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────────┘
Property Inspector File Structure
your-plugin.sdPlugin/
├── manifest.json
├── bin/
│ └── plugin.js ← Compiled backend
├── ui/
│ ├── timer-settings.html ← Property Inspector HTML
│ ├── css/
│ │ └── sdpi.css ← Stream Deck styles
│ └── js/
│ └── sdpi.js ← Stream Deck UI library
└── actions/
└── timer/
├── icon.png ← Action icon
└── icon@2x.png ← Retina icon
Property Inspector Communication Pattern
<!-- ui/timer-settings.html -->
<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" href="css/sdpi.css">
</head>
<body>
<div class="sdpi-wrapper">
<div class="sdpi-item">
<label class="sdpi-item-label">Work Duration</label>
<select id="workDuration" class="sdpi-item-value">
<option value="15">15 minutes</option>
<option value="25">25 minutes</option>
<option value="50">50 minutes</option>
</select>
</div>
</div>
<script src="js/sdpi.js"></script>
<script>
// Connect to Stream Deck
$SD.on("connected", (ev) => {
// Request current settings
$SD.api.getSettings();
});
// Receive settings from plugin
$SD.on("didReceiveSettings", (ev) => {
const settings = ev.payload.settings;
document.getElementById("workDuration").value = settings.workDuration || "25";
});
// When user changes setting, save it
document.getElementById("workDuration").addEventListener("change", (e) => {
$SD.api.setSettings({
workDuration: parseInt(e.target.value)
});
});
</script>
</body>
</html>
Dynamic Image Rendering: Making Keys Come Alive
One of Stream Deck’s most powerful features is updating key images dynamically:
┌─────────────────────────────────────────────────────────────────────────────┐
│ DYNAMIC IMAGE UPDATE FLOW │
│ │
│ Your Code Canvas Base64 Key │
│ │ │ │ │ │
│ │ Create canvas │ │ │ │
│ │──────────────────────► │ │ │
│ │ │ │ │ │
│ │ Draw graphics │ │ │ │
│ │ (text, shapes, │ │ │ │
│ │ progress bars) │ │ │ │
│ │──────────────────────► │ │ │
│ │ │ │ │ │
│ │ │ toDataURL() │ │ │
│ │ │──────────────────────►│ │ │
│ │ │ │ │ │
│ │ │ "data:image/png; │ │ │
│ │ │ base64,iVBORw..." │ │ │
│ │ │◄──────────────────────│ │ │
│ │ │ │ │ │
│ │ setImage(base64) │ │ LCD Update │ │
│ │──────────────────────────────────────────────────────────────────►│ │
│ │ │ │ │ │
│ │ │ │ ┌──────────┐ │ │
│ │ │ │ │ 25:00 │ │ │
│ │ │ │ │ ████░░ │ │ │
│ │ │ │ │ WORK │ │ │
│ │ │ │ └──────────┘ │ │
└─────────────────────────────────────────────────────────────────────────────┘
Key Dimensions and Considerations
Stream Deck Models:
┌──────────────────────────────────────────────────────────────────────────┐
│ │
│ Stream Deck (Original/MK2) Stream Deck Mini Stream Deck XL │
│ ────────────────────────── ──────────────── ────────────── │
│ 15 keys (5x3) 6 keys (3x2) 32 keys (8x4) │
│ 72x72 pixels per key 72x72 pixels 96x96 pixels │
│ │
│ Stream Deck + Stream Deck Neo │
│ ──────────────── ──────────────── │
│ 8 keys + 4 dials 8 keys (4x2) │
│ + touch strip 72x72 pixels │
│ 200x100 pixels │
│ │
└──────────────────────────────────────────────────────────────────────────┘
Icon Specifications:
┌──────────────────────────────────────────────────────────────────────────┐
│ │
│ Standard Keys: Retina Keys (@2x): Touch Strip: │
│ ────────────── ───────────────── ─────────── │
│ 72 x 72 pixels 144 x 144 pixels 800 x 100 pixels │
│ PNG format PNG format PNG format │
│ 24-bit color 24-bit color 24-bit color │
│ Transparent OK Transparent OK Transparent OK │
│ │
└──────────────────────────────────────────────────────────────────────────┘
Concept Summary Table
| Concept Cluster | What You Need to Internalize |
|---|---|
| Plugin Architecture | Two processes (backend + PI), both communicate via WebSocket to Stream Deck app, independently |
| Manifest Configuration | JSON defines everything: actions, icons, entry points, requirements—change manifest, change behavior |
| Actions as Classes | SingletonAction pattern, override event handlers, each action is a self-contained unit |
| Event-Driven Model | Your code responds to events (onKeyDown, onWillAppear), not polling—reactive programming |
| WebSocket Communication | JSON messages over WebSocket, bidirectional, asynchronous—same pattern as real-time web apps |
| Settings Persistence | Global (plugin-wide) vs Action (per-button), automatic save/load, survive app restart |
| Property Inspector | HTML/CSS/JS in a constrained webview, communicates with backend via Stream Deck as intermediary |
| Dynamic Images | Canvas rendering → Base64 → setImage(), 72x72 pixels, real-time updates possible |
| Multi-Action Plugins | One plugin can define multiple action types, each with its own behavior and settings |
| Plugin Lifecycle | Stream Deck spawns plugin at launch, maintains WebSocket, handles profile/device changes |
Deep Dive Reading by Concept
This section maps each concept to specific resources for deeper understanding.
Plugin Architecture & WebSocket Communication
| Concept | Resource | Chapter/Section |
|---|---|---|
| WebSocket fundamentals | The Sockets Networking API by Stevens | Ch. 3: “The WebSocket Protocol” concepts |
| Event-driven architecture | Node.js Design Patterns by Casciaro & Mammino | Ch. 3: “Callbacks and Events” |
| Message passing patterns | Enterprise Integration Patterns by Hohpe & Woolf | Ch. 3: “Messaging Systems” |
| JSON serialization | JavaScript: The Good Parts by Crockford | Ch. 6: “Arrays” & JSON section |
Actions & Event Handling
| Concept | Resource | Chapter/Section |
|---|---|---|
| State machines for UI | The Pragmatic Programmer by Hunt & Thomas | Topic 26: “Programming by Coincidence” |
| Reactive programming | Grokking Reactive Programming (or RxJS docs) | Observable patterns |
| TypeScript decorators | Programming TypeScript by Cherny | Ch. 6: “Advanced Types” (decorators) |
| Async event handling | Node.js Design Patterns by Casciaro & Mammino | Ch. 4: “Asynchronous Control Flow” |
UI Development (Property Inspector)
| Concept | Resource | Chapter/Section |
|---|---|---|
| HTML forms and inputs | HTML5 Canvas by Fulton & Fulton | Form handling concepts |
| CSS in constrained spaces | CSS: The Definitive Guide by Meyer | Responsive design principles |
| Web components | Web Components in Action by Newcome | Custom elements |
| DOM event handling | JavaScript: The Definitive Guide by Flanagan | Ch. 15: “Events” |
Graphics & Dynamic Rendering
| Concept | Resource | Chapter/Section |
|---|---|---|
| Canvas 2D API | HTML5 Canvas by Fulton & Fulton | Ch. 2-4: Drawing basics |
| Efficient rendering | Computer Graphics from Scratch by Gambetta | Optimization techniques |
| Color theory for UIs | The Design of Everyday Things by Norman | Visual feedback principles |
| Animation timing | HTML5 Canvas by Fulton & Fulton | Ch. 9: “Animation” |
Settings & Data Persistence
| Concept | Resource | Chapter/Section |
|---|---|---|
| State management | Effective TypeScript by Vanderkam | Items on type-safe state |
| JSON schema validation | Designing Data-Intensive Applications by Kleppmann | Schema evolution |
| Config management patterns | The Pragmatic Programmer by Hunt & Thomas | Topic 12: “Prototypes and Post-it Notes” |
Essential Reading Order
For maximum comprehension, approach the concepts in this order:
- Foundation (Before starting):
- Stream Deck SDK official docs: Getting Started
- Node.js Design Patterns Ch. 3 (events)
- The Pragmatic Programmer Topic 26 (state machines)
- Core Development (During Project 1-2):
- HTML5 Canvas Ch. 2-4 (for dynamic images)
- Effective TypeScript (for type-safe settings)
- Advanced Patterns (Projects 3+):
- Designing Data-Intensive Applications Ch. 5 (state sync)
- Enterprise Integration Patterns (message passing)
Project 1: Personal Pomodoro Timer
📚 Deep Dive: Complete Project Guide — Full theory, implementation guide, and exercises
- File: ELGATO_STREAM_DECK_PLUGIN_LEARNING_PROJECTS.md
- Programming Language: JavaScript/TypeScript
- Coolness Level: Level 2: Practical but Forgettable
- Business Potential: 2. The “Micro-SaaS / Pro Tool”
- Difficulty: Level 1: Beginner
- Knowledge Area: GUI / Hardware Integration
- Software or Tool: Elgato Stream Deck SDK
- Main Book: “The Pragmatic Programmer” (for state machines context)
What you’ll build: A Stream Deck action that displays a visual countdown timer on the key, with start/pause/reset functionality and configurable work/break intervals.
Why it teaches Stream Deck development: This project forces you to master the complete plugin lifecycle—from action creation to dynamic image updates to settings persistence. You’ll see your timer ticking down directly on the physical button.
Core challenges you’ll face:
- Dynamic key image updates (maps to
setImageAPI and canvas rendering) - State management across key presses (maps to action lifecycle and settings)
- Property Inspector UI (maps to building configuration forms in HTML/JS)
- Timer logic with Stream Deck events (maps to event-driven architecture)
Resources for key challenges:
- Stream Deck SDK Getting Started - Official scaffolding walkthrough
- Settings Guide - How to persist timer configurations
Key Concepts:
- Plugin scaffolding: Stream Deck CLI Documentation - Elgato
- Action classes: GitHub SDK - SingletonAction - Elgato
- Dynamic images: Plugin WebSocket API - Elgato
- State machines: The Pragmatic Programmer (Ch. 26: “Programming by Coincidence”) - Hunt & Thomas
Difficulty: Beginner Time estimate: Weekend Prerequisites: Basic JavaScript/TypeScript, Node.js fundamentals
Real world outcome:
- A fully functional Pomodoro timer displayed on your Stream Deck
- Watch the countdown update in real-time on the physical LCD key
- Audible/visual alerts when work/break periods end
- Configurable intervals through the Property Inspector UI
Learning milestones:
- First milestone: Plugin loads and displays a static action icon in Stream Deck
- Second milestone: Key press starts/stops a timer with console logging
- Third milestone: Timer countdown renders dynamically on the key itself
- Final milestone: Full Property Inspector with saved settings and notification sounds
What you’ll actually see:
Stream Deck Key (72x72 pixels):
┌─────────────────────┐
│ │
│ ┌─────────┐ │
│ │ 24:37 │ │ <- Time remaining in large text
│ └─────────┘ │
│ FOCUS │ <- Current mode (FOCUS/BREAK)
│ ●────────○ │ <- Progress bar
└─────────────────────┘
States:
- IDLE: Shows "START" with tomato icon, gray background
- RUNNING: Shows countdown "MM:SS", green border, progress bar fills
- PAUSED: Shows paused time, yellow background, pulsing effect
- BREAK: Shows break countdown, blue background, coffee icon
Key States Visual Progression:
┌─────────────────────────────────────────────────────────────────┐
│ YOUR STREAM DECK KEY │
│ │
│ IDLE STATE WORKING STATE BREAK STATE │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ │ │ 24:37 │ │ 04:22 │ │
│ │ START │ --> │ ████████ │ --> │ ██████░░ │ │
│ │ 25:00 │ │ WORK │ │ BREAK │ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ (Gray) (Red/Orange) (Green/Blue) │
└─────────────────────────────────────────────────────────────────┘
Property Inspector UI (appears when you click the gear icon):
┌─────────────────────────────────────────────┐
│ Pomodoro Timer Settings │
├─────────────────────────────────────────────┤
│ Work Duration: [25] minutes │
│ Break Duration: [5 ] minutes │
│ Long Break: [15] minutes │
│ Sessions Before Long Break: [4] │
│ │
│ ☑ Play sound on completion │
│ ☑ Auto-start break after work │
│ ☐ Auto-start work after break │
│ │
│ Sound: [Chime ▼] │
└─────────────────────────────────────────────┘
Interaction Flow:
- First press: Timer starts counting down from configured work duration
- Press during countdown: Pauses the timer (press again to resume)
- Long press (hold 2s): Resets timer to initial state
- Automatic transition: When work ends, plays sound and switches to break mode
The Core Question You’re Answering
“How do I make a Stream Deck key update its display dynamically based on internal state, while persisting user preferences across restarts?”
This project answers the fundamental question of real-time visual feedback in Stream Deck plugins. By the end, you’ll understand the complete data flow: user input -> state change -> visual update -> persistence.
Concepts You Must Understand First
Before writing any code, ensure you understand these foundations:
| Concept | Why It Matters | Where to Learn |
|---|---|---|
| Plugin Lifecycle Events | onWillAppear and onKeyDown are your entry points for all interaction. Without understanding when these fire, your timer will behave unpredictably. |
Stream Deck SDK docs, Node.js Design Patterns Ch. 3 (Event-Driven Architecture) |
| State Machines | A Pomodoro timer has distinct states (IDLE, RUNNING, PAUSED, BREAK). State machines prevent “impossible” states like being paused and running simultaneously. | The Pragmatic Programmer Topic 26, Refactoring Ch. 10 (State Pattern) |
| Canvas Rendering Basics | Stream Deck keys are 72x72 pixel images. You’ll generate these dynamically using HTML5 Canvas or node-canvas. | HTML5 Canvas by Fulton & Fulton Ch. 2-4 |
| Settings Persistence | User preferences must survive plugin restarts. Stream Deck provides setSettings/getSettings for this. |
Stream Deck SDK Settings Guide |
| setInterval/Timers in Node.js | Your countdown runs via JavaScript timers. Understand how they interact with the event loop. | Node.js Design Patterns Ch. 2 (Asynchronous Control Flow) |
Deep Dive: The State Machine
Your Pomodoro timer has exactly four states:
┌──────────────┐
│ IDLE │
│ (waiting to │
│ start) │
└──────┬───────┘
│ KEY_DOWN
▼
┌──────────────────────────────────────┐
│ │
▼ │
┌──────────────┐ KEY_DOWN ┌──────────────┐│
│ RUNNING │◄─────────────►│ PAUSED ││
│ (counting │ │ (timer ││
│ down) │ │ stopped) ││
└──────┬───────┘ └──────────────┘│
│ TIMER_COMPLETE │
▼ │
┌──────────────┐ │
│ BREAK │───────────────────────────────┘
│ (resting) │ TIMER_COMPLETE or LONG_PRESS
└──────────────┘
Questions to Guide Your Design
Before coding, answer these questions on paper:
-
State Tracking: How will you track timer state between key presses? (Hint: class instance variables vs. Stream Deck settings)
-
Persistence: What happens when the user closes and reopens Stream Deck? Should the timer resume or reset? How do you decide?
-
Canvas Rendering: How do you render time remaining on a 72x72 pixel canvas? What font size is readable? How do you center text?
-
Transitions: How will you handle work/break transitions? Should they be automatic or require user confirmation?
-
Multiple Instances: What if the user drags two Pomodoro actions onto different keys? Should they share state or be independent?
-
Error Handling: What happens if your timer callback throws an error? Does the timer stop? Can you recover?
Thinking Exercise
Before writing code, trace through this scenario mentally:
SCENARIO: User has a 25-minute Pomodoro timer. They work for 10 minutes,
pause for 5 minutes (coffee break), then resume.
Time 0:00 - User presses key
→ What event fires? _______________
→ What state do you enter? _______________
→ What does the key display? _______________
Time 10:00 - User presses key to pause
→ Timer shows "15:00" remaining
→ What do you save? _______________
→ What happens to setInterval? _______________
Time 15:00 - User presses key to resume (after 5 min pause)
→ Should the timer still show "15:00"? _______________
→ How do you restart the countdown? _______________
Time 30:00 - Timer reaches 00:00 (15 min later)
→ What event triggers? _______________
→ What sound plays? _______________
→ What state do you transition to? _______________
The Interview Questions They’ll Ask
After completing this project, you should be able to answer:
- “What is the Stream Deck plugin lifecycle?”
- Expected answer: Explain
onWillAppear,onWillDisappear,onKeyDown,onKeyUp, and when each fires
- Expected answer: Explain
- “How do you persist state between plugin restarts?”
- Expected answer: Discuss
setSettings/getSettings, what belongs in settings vs. runtime state, and the serialization format
- Expected answer: Discuss
- “Explain how WebSocket communication works in Stream Deck plugins.”
- Expected answer: The plugin runs in Node.js, communicates with Stream Deck app via WebSocket. SDK abstracts this but you should know
streamDeck.connect()establishes the connection
- Expected answer: The plugin runs in Node.js, communicates with Stream Deck app via WebSocket. SDK abstracts this but you should know
- “How would you test a Stream Deck plugin?”
- Expected answer: Unit tests for state machine logic, mocking the SDK for integration tests, manual testing with real hardware
- “Your timer drifts by a few seconds over an hour. Why?”
- Expected answer:
setIntervalisn’t guaranteed to fire exactly on time. Discuss calculating elapsed time from timestamps vs. decrementing a counter
- Expected answer:
Hints in Layers
If you get stuck, reveal hints progressively:
Hint 1: Timer State Management (click to reveal)
Your action class needs these instance variables:
private state: 'idle' | 'running' | 'paused' | 'break' = 'idle';
private remainingSeconds: number = 0;
private intervalId: NodeJS.Timeout | null = null;
private settings: PomodoroSettings = { workMinutes: 25, breakMinutes: 5 };
The key insight: state and remainingSeconds are runtime state (lost on restart), while settings are persisted via the SDK.
Hint 2: Canvas Rendering for Countdown (click to reveal)
private renderTimer(minutes: number, seconds: number): string {
const canvas = document.createElement('canvas');
canvas.width = 144; // 2x for retina displays
canvas.height = 144;
const ctx = canvas.getContext('2d')!;
// Background
ctx.fillStyle = this.state === 'break' ? '#3498db' : '#2ecc71';
ctx.fillRect(0, 0, 144, 144);
// Time text
ctx.fillStyle = 'white';
ctx.font = 'bold 48px Arial';
ctx.textAlign = 'center';
ctx.textBaseline = 'middle';
ctx.fillText(`${minutes}:${seconds.toString().padStart(2, '0')}`, 72, 72);
return canvas.toDataURL('image/png');
}
Hint 3: Settings Persistence Pattern (click to reveal)
// In onWillAppear - load saved settings
override onWillAppear(ev: WillAppearEvent<PomodoroSettings>): void {
this.settings = ev.payload.settings ?? { workMinutes: 25, breakMinutes: 5 };
this.remainingSeconds = this.settings.workMinutes * 60;
this.updateDisplay();
}
// In Property Inspector - save settings
document.getElementById('save-btn').addEventListener('click', () => {
const settings = {
workMinutes: parseInt(document.getElementById('work-input').value),
breakMinutes: parseInt(document.getElementById('break-input').value)
};
$SD.setSettings(settings); // SDK method
});
Hint 4: Sound Notification Approach (click to reveal)
Stream Deck plugins run in Node.js, which doesn’t have Audio API. Options:
- System notification with sound: Use
node-notifierpackage - Play audio file: Use
play-soundornode-wav-playerpackages - Stream Deck built-in: Use
streamDeck.system.openUrl('file://...')with audio file (hacky)
Recommended approach:
import player from 'play-sound';
private playNotificationSound(): void {
const soundPath = path.join(__dirname, 'assets', 'chime.mp3');
player().play(soundPath, (err) => {
if (err) console.error('Sound playback failed:', err);
});
}
Books That Will Help
| Topic | Book | Chapter/Section | Why It Helps |
|---|---|---|---|
| State machine design | The Pragmatic Programmer | Topic 26: “How to Balance Resources” | Teaches clean state transitions and resource management |
| Event-driven architecture | Node.js Design Patterns | Ch. 3: “Callbacks and Events” | Stream Deck plugins are entirely event-driven |
| Canvas rendering | HTML5 Canvas by Fulton & Fulton | Ch. 2-4 | Drawing dynamic images for the key display |
| TypeScript for settings | Effective TypeScript | Items 29-37 | Type-safe configuration objects |
| Timer precision | You Don’t Know JS: Async & Performance | Ch. 1 | Understanding JavaScript timing guarantees |
| Clean code structure | Refactoring by Fowler | Ch. 10: “Organizing Data” | Keeping state management clean |
Project 2: System Monitor Dashboard
📚 Deep Dive: Complete Project Guide — Full theory, implementation guide, and exercises
- File: stream_deck_system_monitor.md
- Main Programming Language: TypeScript
- Alternative Programming Languages: JavaScript, Python
- Coolness Level: Level 2: Practical but Forgettable
- Business Potential: Level 2: The “Micro-SaaS / Pro Tool”
- Difficulty: Level 2: Intermediate (The Developer)
- Knowledge Area: Plugin Development, System Monitoring
- Software or Tool: Elgato Stream Deck SDK, Node.js
- Main Book: “Node.js Design Patterns” by Casciaro & Mammino
What you’ll build: A multi-action plugin that displays real-time CPU usage, memory consumption, and disk space on dedicated Stream Deck keys with color-coded thresholds.
Why it teaches Stream Deck development: This project introduces multi-action plugins, polling external data, and visual feedback design. You’ll learn how to create plugins that continuously update based on system state.
Core challenges you’ll face:
- Multiple action types in one plugin (maps to manifest action definitions)
- Polling system metrics from Node.js (maps to
systeminformationlibrary integration) - Color-coded visual thresholds (maps to programmatic image generation)
- Efficient update intervals (maps to performance and battery considerations)
Resources for key challenges:
- Manifest Reference - Defining multiple actions
- systeminformation npm package - Cross-platform system data
Key Concepts:
- Multi-action manifests: Manifest Documentation - Elgato
- Node.js system APIs: systeminformation - npm
- Canvas image generation: node-canvas - npm
- Event polling patterns: Node.js Design Patterns (Ch. 3: “Callbacks and Events”) - Casciaro & Mammino
Difficulty: Beginner-Intermediate Time estimate: 1 week Prerequisites: JavaScript/TypeScript, basic understanding of system metrics
Real world outcome:
- Live CPU percentage displayed on one key (turns red above 80%)
- Memory usage on another key (color gradients based on pressure)
- Disk space indicator that warns when storage is low
- A genuinely useful productivity tool you’ll actually use daily
┌─────────────────────────────────────────────────────────────────────────────┐
│ STREAM DECK SYSTEM MONITOR DASHBOARD │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ Your Stream Deck with multiple system monitoring keys: │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ ████████ │ │ ██████ │ │ ███ │ │ ████████ │ │ OTHER │ │
│ │ ████████ │ │ ██████ │ │ ███ │ │ ████████ │ │ KEY │ │
│ │ │ │ │ │ │ │ │ │ │ │
│ │ CPU 23% │ │ RAM 67% │ │ DISK 89% │ │ NET ↑↓ │ │ │ │
│ │ [GREEN] │ │ [YELLOW] │ │ [RED] │ │ [BLUE] │ │ │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ │ │ │ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │ │ │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
│ │
├─────────────────────────────────────────────────────────────────────────────┤
│ COLOR THRESHOLD LEGEND │
│ │
│ CPU Key States: │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ ▓▓▓▓▓▓▓▓ │ │ ▓▓▓▓▓▓▓▓ │ │ ▓▓▓▓▓▓▓▓ │ │
│ │ 23% │ │ 67% │ │ 89% │ │
│ │ GREEN │ │ YELLOW │ │ RED │ │
│ │ (0-50%) │ │ (51-80%) │ │ (81-100%)│ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ │
│ Gauge Visualization (72x72 canvas): │
│ ┌──────────────────────────────────────┐ │
│ │ ╭─────────────╮ │ │
│ │ ╱ ╲ ╱ ╲ │ │
│ │ │ ▌ │ │ <- Arc gauge fills clockwise │
│ │ ╲ ╱ ╲ ╱ │ │
│ │ ╰─────────────╯ │ │
│ │ 67% │ <- Percentage in center │
│ │ RAM │ <- Label below │
│ └──────────────────────────────────────┘ │
│ │
├─────────────────────────────────────────────────────────────────────────────┤
│ PROPERTY INSPECTOR UI │
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ System Monitor Configuration │ │
│ ├─────────────────────────────────────────────────────────────────────┤ │
│ │ │ │
│ │ Polling Interval │ │
│ │ ┌─────────────────────────────────────────────────────┐ │ │
│ │ │ 2000 v │ ms │ │
│ │ └─────────────────────────────────────────────────────┘ │ │
│ │ (i) Lower values = more CPU usage. Recommended: 1000-5000ms │ │
│ │ │ │
│ │ ───────────────────────────────────────────────────────────── │ │
│ │ │ │
│ │ Warning Threshold (Yellow) │ │
│ │ ┌─────────────────────────────────────────────────────┐ │ │
│ │ │ 50 v │ % │ │
│ │ └─────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ Critical Threshold (Red) │ │
│ │ ┌─────────────────────────────────────────────────────┐ │ │
│ │ │ 80 v │ % │ │
│ │ └─────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ───────────────────────────────────────────────────────────── │ │
│ │ │ │
│ │ Display Style │ │
│ │ ( ) Percentage Only │ │
│ │ (x) Gauge with Percentage │ │
│ │ ( ) Bar Graph │ │
│ │ │ │
│ │ ───────────────────────────────────────────────────────────── │ │
│ │ │ │
│ │ [x] Show label below value │ │
│ │ [x] Animate threshold transitions │ │
│ │ [ ] Flash on critical │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Learning milestones:
- First milestone: Single CPU action displays static percentage
- Second milestone: Real-time updates with configurable polling interval
- Third milestone: Color-coded thresholds with smooth gradients
- Final milestone: Complete dashboard with multiple metrics and Property Inspector configuration
The Core Question You’re Answering
“How do I create a Stream Deck plugin with multiple action types that continuously poll external data sources and display that information visually with dynamic, color-coded feedback?”
This project forces you to solve the fundamental challenge of multi-action plugin architecture. Unlike Project 1 where you had one action doing one thing, here you’re building a dashboard where CPU, RAM, and Disk each have their own action class, their own settings, their own visual representations—but they all share a common polling infrastructure and rendering pipeline.
The deeper question is about resource management: How do you balance update frequency against system load? How do you avoid hammering the CPU just to display CPU usage? This tension between real-time accuracy and efficient polling is a pattern you’ll encounter in every monitoring tool you ever build.
Concepts You Must Understand First
Before writing any code, you need solid mental models for these concepts:
1. Multi-Action Manifest Configuration
Your manifest.json must define multiple actions within a single plugin. Each action (CPU, RAM, Disk) needs its own UUID, icon, and action class. Understanding how Stream Deck routes events to the correct action handler is essential.
- Stream Deck SDK Manifest Reference: Manifest Documentation
- Node.js Design Patterns (Ch. 8: “Structural Design Patterns”) - How to organize code for multiple similar-but-different components
2. Polling Patterns and Intervals
Polling is the act of repeatedly asking “what’s the current value?” at fixed intervals. You’ll use setInterval() but need to understand the tradeoffs:
- Too fast (100ms) = high CPU overhead, battery drain
- Too slow (10s) = stale data, missed spikes
-
Just right (1-3s) = acceptable for human perception
- Node.js Design Patterns (Ch. 3: “Callbacks and Events”) - Event loop behavior with timers
- Node.js Design Patterns (Ch. 11: “Advanced Recipes”) - Managing async operations over time
3. Node.js System Information APIs
The systeminformation library provides cross-platform access to CPU load, memory usage, disk space, network stats, and more. Understanding its async API and the shape of returned data structures is critical.
- systeminformation npm package - API reference and examples
- Node.js Design Patterns (Ch. 5: “Asynchronous Control Flow Patterns with Promises and Async/Await”)
4. Color-Coded Thresholds and Gradient Rendering You’re not just showing numbers—you’re encoding meaning through color. This requires understanding:
- HSL color space (why it’s better for gradients than RGB)
- Linear interpolation between colors
-
Threshold logic (if value > X, use color Y)
- HTML5 Canvas by Steve Fulton (Ch. 4: “Images on the Canvas”) - Color manipulation fundamentals
5. Canvas Drawing for Gauges and Meters
Stream Deck keys are 72x72 or 144x144 pixels. You’ll draw arc gauges, progress bars, or numeric displays using the HTML Canvas API (via node-canvas). This is programmatic image generation, not static assets.
- node-canvas npm package - Canvas in Node.js
- HTML5 Canvas by Steve Fulton (Ch. 2: “Drawing on the Canvas”, Ch. 5: “Making Things Move”)
Questions to Guide Your Design
Work through these questions before coding. They’ll surface the architectural decisions you need to make:
Manifest & Architecture
- How do you define multiple actions in
manifest.json? What makes each action unique? - Should CPU, RAM, and Disk share a base class, or should they be completely independent?
- How does Stream Deck route events (
willAppear,keyDown) to the correct action instance? - Can you have multiple instances of the same action (e.g., two CPU keys showing different cores)?
Polling & Data Flow
- Where should the polling loop live—in each action instance, or in a shared service?
- What polling interval balances real-time feel against CPU overhead? (Hint: 1000-2000ms is typical)
- How do you stop polling when an action is removed from the deck?
- What happens if
systeminformationthrows an error during a poll cycle?
Visual Rendering
- How do you render a CPU percentage meter on a 72x72 canvas?
- Should you use arc gauges, bar graphs, or just large numbers? (User preference via Property Inspector?)
- How do you calculate the color for 67% usage? (Linear interpolation between thresholds)
- How do you handle high CPU states—just color change, or also animation/flashing?
Property Inspector
- What settings should be configurable per-action? (Polling interval, thresholds, display style)
- How do global settings differ from action-specific settings?
- How do you validate that warning threshold < critical threshold?
Thinking Exercise
Before coding, trace through this mental simulation of how polling and rendering interact:
MENTAL MODEL: Polling + Rendering Pipeline
==========================================
T=0ms: Plugin initializes
├── Create CpuAction, RamAction, DiskAction instances
├── Each action calls startPolling() in onWillAppear
└── Each sets up: setInterval(pollAndRender, 2000)
T=2000ms: First poll cycle fires
├── CpuAction.pollAndRender():
│ ├── const cpu = await si.currentLoad() // ~50ms
│ ├── const percentage = cpu.currentLoad // e.g., 67
│ ├── const color = calculateColor(67, thresholds) // yellow
│ ├── const image = renderGauge(67, color) // Canvas magic
│ └── this.setImage(image) // Send to Stream Deck
│
├── RamAction.pollAndRender():
│ ├── const mem = await si.mem() // ~30ms
│ ├── const percentage = (mem.used / mem.total) * 100
│ └── ... render and send
│
└── DiskAction.pollAndRender():
├── const disk = await si.fsSize() // ~100ms
└── ... render and send
T=4000ms: Second poll cycle fires
└── Same flow, but values may have changed
QUESTION: What happens if si.currentLoad() takes 500ms?
ANSWER: The interval still fires every 2000ms, but you might get overlapping
calls if you're not careful. Consider using a lock or Promise chain.
QUESTION: What happens when user removes the CPU key from their deck?
ANSWER: onWillDisappear fires -> clearInterval(this.pollTimer) -> cleanup
Walk through this diagram mentally:
- What would happen if you forgot to clear the interval on
willDisappear? (Memory leak, zombie updates) - What if two CPU keys are added? (Each gets its own polling loop—is that wasteful?)
- How would you optimize to share one poll across all actions? (Event emitter pattern)
The Interview Questions They’ll Ask
These are real questions you might face when discussing Stream Deck plugin development or general monitoring system design:
Plugin Architecture
- “How do you handle multiple actions in one Stream Deck plugin, and how does event routing work?”
- Expected answer: Explain manifest action definitions, unique UUIDs, and how the SDK routes events based on action context.
- “What’s the difference between global settings and action settings in Stream Deck plugins?”
- Expected answer: Global settings persist across all actions (set once, affect everything). Action settings are per-instance (each key can have different thresholds).
System Design
- “How do you optimize polling for performance when monitoring multiple metrics?”
- Expected answer: Discuss shared polling vs. per-action polling, debouncing, caching recent values, and adjusting intervals based on visibility.
- “Your CPU monitor is causing 5% CPU usage itself. How do you debug and optimize this?”
- Expected answer: Profile with Chrome DevTools, check polling frequency, batch
systeminformationcalls, use requestAnimationFrame concepts for rendering.
- Expected answer: Profile with Chrome DevTools, check polling frequency, batch
Error Handling
- “What happens if the system information library fails during a poll? How do you handle it gracefully?”
- Expected answer: try/catch around async calls, display “N/A” or last known value, implement exponential backoff, log errors for debugging.
Visual Design
- “How do you decide what color to show for 67% CPU usage if your thresholds are 50% (yellow) and 80% (red)?”
- Expected answer: Linear interpolation between yellow and red, or step function at discrete thresholds. Discuss HSL vs RGB for smooth gradients.
Hints in Layers
Reveal these progressively as you work through the project:
Hint 1: Getting System Information
The systeminformation library is your friend. Install it and explore the API:
import si from 'systeminformation';
// CPU usage (percentage 0-100)
const cpu = await si.currentLoad();
console.log(cpu.currentLoad); // e.g., 45.2
// Memory (bytes)
const mem = await si.mem();
console.log((mem.used / mem.total) * 100); // percentage used
// Disk (array of filesystems)
const disks = await si.fsSize();
console.log(disks[0].use); // percentage used for first disk
Hint 2: Polling with setInterval Set up a polling loop that cleans up properly:
class CpuAction extends Action {
private pollTimer?: NodeJS.Timeout;
override onWillAppear(): void {
this.startPolling();
}
override onWillDisappear(): void {
this.stopPolling();
}
private startPolling(): void {
// Poll immediately, then every 2 seconds
this.pollAndRender();
this.pollTimer = setInterval(() => this.pollAndRender(), 2000);
}
private stopPolling(): void {
if (this.pollTimer) {
clearInterval(this.pollTimer);
this.pollTimer = undefined;
}
}
private async pollAndRender(): Promise<void> {
try {
const cpu = await si.currentLoad();
await this.renderGauge(cpu.currentLoad);
} catch (error) {
console.error('Failed to get CPU info:', error);
await this.renderError();
}
}
}
Hint 3: Canvas Gauge Rendering Draw an arc gauge that fills based on percentage:
import { createCanvas } from 'canvas';
function renderGauge(percentage: number, thresholds: Thresholds): string {
const size = 144; // 144x144 for @2x retina
const canvas = createCanvas(size, size);
const ctx = canvas.getContext('2d');
// Background
ctx.fillStyle = '#1a1a2e';
ctx.fillRect(0, 0, size, size);
// Arc settings
const centerX = size / 2;
const centerY = size / 2;
const radius = size * 0.35;
const startAngle = 0.75 * Math.PI; // 7 o'clock
const endAngle = 2.25 * Math.PI; // 5 o'clock
const arcLength = endAngle - startAngle;
// Background arc (gray)
ctx.beginPath();
ctx.arc(centerX, centerY, radius, startAngle, endAngle);
ctx.strokeStyle = '#333';
ctx.lineWidth = 12;
ctx.lineCap = 'round';
ctx.stroke();
// Value arc (colored by threshold)
const valueAngle = startAngle + (percentage / 100) * arcLength;
ctx.beginPath();
ctx.arc(centerX, centerY, radius, startAngle, valueAngle);
ctx.strokeStyle = getColorForValue(percentage, thresholds);
ctx.stroke();
// Percentage text
ctx.fillStyle = '#fff';
ctx.font = 'bold 32px Arial';
ctx.textAlign = 'center';
ctx.textBaseline = 'middle';
ctx.fillText(`${Math.round(percentage)}%`, centerX, centerY);
return canvas.toDataURL(); // Returns base64 data URL for setImage()
}
Hint 4: Color Gradients Based on Thresholds Calculate colors that smoothly transition between thresholds:
interface Thresholds {
warning: number; // e.g., 50
critical: number; // e.g., 80
}
function getColorForValue(value: number, thresholds: Thresholds): string {
// Below warning: green
if (value < thresholds.warning) {
return '#4ade80'; // green-400
}
// Between warning and critical: interpolate yellow to orange
if (value < thresholds.critical) {
const t = (value - thresholds.warning) / (thresholds.critical - thresholds.warning);
return interpolateColor('#facc15', '#f97316', t); // yellow to orange
}
// Above critical: red
return '#ef4444'; // red-500
}
function interpolateColor(color1: string, color2: string, t: number): string {
// Parse hex to RGB
const r1 = parseInt(color1.slice(1, 3), 16);
const g1 = parseInt(color1.slice(3, 5), 16);
const b1 = parseInt(color1.slice(5, 7), 16);
const r2 = parseInt(color2.slice(1, 3), 16);
const g2 = parseInt(color2.slice(3, 5), 16);
const b2 = parseInt(color2.slice(5, 7), 16);
// Interpolate
const r = Math.round(r1 + (r2 - r1) * t);
const g = Math.round(g1 + (g2 - g1) * t);
const b = Math.round(b1 + (b2 - b1) * t);
return `#${r.toString(16).padStart(2, '0')}${g.toString(16).padStart(2, '0')}${b.toString(16).padStart(2, '0')}`;
}
Books That Will Help
| Topic | Book | Chapter/Section | Why It Helps |
|---|---|---|---|
| Multi-action architecture | Node.js Design Patterns | Ch. 8: Structural Patterns | Organizing code for multiple action classes sharing common functionality |
| Polling and timers | Node.js Design Patterns | Ch. 3: Callbacks and Events | Understanding event loop behavior with setInterval |
| Async data fetching | Node.js Design Patterns | Ch. 5: Promises and Async/Await | Clean async code for systeminformation calls |
| Canvas graphics | HTML5 Canvas | Ch. 2, 4, 5: Drawing, Images, Animation | Rendering gauges, gradients, and meters |
| Color theory for UI | Refactoring UI | “Color” section | Choosing effective threshold colors |
| TypeScript patterns | Effective TypeScript | Item 30: Type inference | Typing your action settings and state |
| Error handling | Effective TypeScript | Item 46: unknown vs any | Gracefully handling polling failures |
| Resource cleanup | Node.js Design Patterns | Ch. 3: Callbacks and Events | Properly clearing intervals on cleanup |
Project 3: Smart Home Controller (Home Assistant/MQTT)
📚 Deep Dive: Complete Project Guide — Full theory, implementation guide, and exercises
- File: smart_home_controller_stream_deck.md
- Main Programming Language: TypeScript
- Alternative Programming Languages: JavaScript, Python, Go
- Coolness Level: Level 3: Genuinely Clever
- Business Potential: Level 2: The “Micro-SaaS / Pro Tool”
- Difficulty: Level 2: Intermediate (The Developer)
- Knowledge Area: IoT, Smart Home, WebSocket
- Software or Tool: Home Assistant, Stream Deck, MQTT
- Main Book: Designing Data-Intensive Applications by Martin Kleppmann
What you’ll build: A plugin that controls smart home devices—toggling lights, adjusting thermostat, showing sensor states—with bidirectional state synchronization.
Why it teaches Stream Deck development: This project teaches external API integration, WebSocket bidirectional communication, authentication handling, and real-time state reflection. The key icons will update when you control devices from other apps too.
Core challenges you’ll face:
- OAuth/API token authentication (maps to secure credential storage)
- WebSocket connections to external services (maps to Home Assistant WebSocket API)
- Bidirectional state sync (maps to reflecting device state changes on keys)
- Error handling and reconnection (maps to robust plugin architecture)
Resources for key challenges:
- Home Assistant WebSocket API - Official HA docs
- Stream Deck Settings for Secrets - Secure token storage
Key Concepts:
- WebSocket client connections: ws npm package - npm
- OAuth flows in desktop apps: Home Assistant Long-Lived Tokens - HA Docs
- State synchronization: Designing Data-Intensive Applications (Ch. 5: “Replication”) - Kleppmann
- Error handling patterns: Effective TypeScript (Item 46: “Use unknown Instead of any”) - Vanderkam
Difficulty: Intermediate Time estimate: 1-2 weeks Prerequisites: JavaScript/TypeScript, basic understanding of REST/WebSocket APIs, a Home Assistant instance (or MQTT broker)
Real world outcome:
- Press a key to toggle your living room lights
- Thermostat key shows current temperature and adjusts on press/dial
- Door sensor key turns red when door is open
- All keys update in real-time when devices change from any source
+------------------+ +------------------+ +------------------+
| Living Room | | Thermostat | | Front Door |
| | | | | |
| [BULB] | | 72°F | | [DOOR] |
| OFF | | [====| ] | | CLOSED |
| ~~~~~~~~~~~~ | | Target: 70°F | | ~~~~~~~~~~~~ |
| Tap to Toggle | | Dial to Adjust | | Sensor Status |
+------------------+ +------------------+ +------------------+
| | |
v v v
+------------------+ +------------------+ +------------------+
| Living Room | | Thermostat | | Front Door |
| | | | | |
| [BULB] | | 70°F | | [DOOR] |
| ON | | [==| ] | | OPEN |
| ~~~~~~~~~~~~ | | Target: 70°F | | ~~~~~~~~~~~~ |
| Tap to Toggle | | Cooling... | | !! ALERT !! |
+------------------+ +------------------+ +------------------+
(Yellow) (Blue) (Red)
Bidirectional behavior: When you toggle the living room light from your phone’s Home Assistant app, the Stream Deck key automatically updates from OFF to ON within seconds. The plugin maintains a persistent WebSocket subscription to Home Assistant’s state change events, ensuring your physical control surface always reflects the true state of your smart home.
Learning milestones:
- First milestone: Connect to Home Assistant and list available entities
- Second milestone: Single light toggle action with icon state reflection
- Third milestone: Property Inspector for selecting entities and configuring actions
- Final milestone: Multiple device types with bidirectional sync and error recovery
The Core Question You’re Answering
How do I create bidirectional communication between Stream Deck and external services with real-time state synchronization?
This project forces you to manage nested WebSocket connections—your plugin connects to Stream Deck via WebSocket, and simultaneously maintains a separate WebSocket connection to Home Assistant. You must handle authentication, state subscriptions, and graceful degradation when either connection fails. This is the pattern behind professional IoT control surfaces.
The hardest conceptual leap: your plugin isn’t the source of truth—Home Assistant is. When you press a key to toggle a light, you’re not toggling state locally then hoping it works. You’re sending a command, waiting for confirmation, and updating the UI only when the external service confirms the change.
Concepts You Must Understand First
Before writing code, ensure you understand these foundational concepts:
1. WebSocket Client Connections (Nested WebSockets) Your plugin is already a WebSocket server (Stream Deck connects to it), but now you also need to be a WebSocket client connecting outbound to Home Assistant. This creates a “WebSocket proxy” pattern where your plugin sits between two WebSocket endpoints.
- Designing Data-Intensive Applications (Ch. 11: “Stream Processing”) - Kleppmann: Understand event-driven architectures and message passing between systems
- ws npm package documentation - The Node.js WebSocket library you’ll use for the Home Assistant connection
2. OAuth/Long-Lived Token Authentication Home Assistant uses long-lived access tokens rather than full OAuth flows. You need to understand:
- How tokens are generated in Home Assistant (User Profile > Long-Lived Access Tokens)
- How to securely store tokens in plugin settings (never hardcode!)
-
Token rotation and expiration handling
- Home Assistant Authentication API - Official documentation
- Effective TypeScript (Item 42: “Understand Type Narrowing”) - Vanderkam: Properly typing authentication responses
3. Bidirectional State Synchronization Patterns The key challenge: when a light turns on, how does your plugin know? You must subscribe to Home Assistant’s state change events and update your key images reactively. This is the classic distributed systems problem: your UI, the real device, and HA’s state can temporarily disagree.
- Designing Data-Intensive Applications (Ch. 5: “Replication”) - Kleppmann: Leader/follower replication concepts apply to state sync
- Designing Data-Intensive Applications (Ch. 12: “The Future of Data Systems”) - Kleppmann: Subscribing to change streams
4. Error Handling and Reconnection Strategies Network connections fail. Home Assistant restarts. Your plugin must handle:
- Initial connection failures (wrong URL, wrong token)
- Mid-session disconnections
- Exponential backoff for reconnection attempts
-
User feedback during connection issues
- Node.js Design Patterns (Ch. 3: “Callbacks and Events”) - Casciaro & Mammino: Event-driven error handling
- Node.js Design Patterns (Ch. 11: “Advanced Recipes”) - Reconnection patterns
- Effective TypeScript (Item 46: “Use unknown Instead of any for Values of Unknown Type”) - Vanderkam: Safe error type handling
5. Entity State Representation Home Assistant entities have complex state objects. A light might be:
{
"entity_id": "light.living_room",
"state": "on",
"attributes": {
"brightness": 255,
"color_temp": 370,
"friendly_name": "Living Room Light"
}
}
You must map this to visual representations on 72x72 pixel keys.
Questions to Guide Your Design
Work through these questions before implementing:
Authentication & Security
- How do you securely store API tokens in settings? (Use Stream Deck’s global settings, never localStorage or hardcoded values)
- What happens if the user enters an invalid token? (Graceful error message, not a crash)
- Should you validate the token on Property Inspector save or on plugin load?
Connection Management
- How do you maintain two WebSocket connections simultaneously? (Event emitters, connection state machines)
- What’s your reconnection strategy? (Exponential backoff: 1s, 2s, 4s, 8s, max 30s)
- How do you notify the user when Home Assistant is unreachable? (Key icon changes, error states)
State Synchronization
- What happens if Home Assistant goes offline while your plugin is running?
- How do you update the key image when device state changes externally (from phone app)?
- How do you handle stale state when reconnecting after a disconnect?
- What if multiple Stream Deck keys control the same entity?
Entity Selection
- How does the user select which entity each key controls? (Property Inspector with entity dropdown)
- How do you fetch the list of available entities for the dropdown?
- Should you cache the entity list or fetch fresh each time?
Thinking Exercise
Trace this complete flow through your system:
User toggles light from phone app
|
v
Home Assistant receives command
|
v
Home Assistant updates internal state
|
v
Home Assistant broadcasts state_changed event via WebSocket
|
v
Your plugin receives WebSocket message
|
v
Your plugin parses entity_id and new state
|
v
Your plugin finds which Stream Deck key(s) control this entity
|
v
Your plugin generates new key image (bulb icon, ON state, yellow background)
|
v
Your plugin calls setImage() via Stream Deck SDK
|
v
Stream Deck hardware displays updated key
Question: At which step could things go wrong? What error handling do you need at each stage?
The Interview Questions They’ll Ask
After building this project, you should be able to answer:
1. “How do you implement bidirectional state sync between a desktop app and an IoT service?”
Strong answer: “I establish a persistent WebSocket connection and subscribe to state change events. When the remote service state changes, I receive an event and update my local representation. When the user takes action locally, I send a command and optimistically update the UI while waiting for the authoritative state confirmation from the server. I handle the case where my optimistic update is wrong—for example, if the light fails to turn on, I revert the icon.”
2. “What’s your reconnection strategy for WebSocket failures?”
Strong answer: “I use exponential backoff with jitter. First retry at 1 second, then 2, 4, 8, up to a maximum of 30 seconds. I add random jitter (plus or minus 20%) to prevent thundering herd problems if many clients reconnect simultaneously. I also distinguish between recoverable errors (network timeout) and fatal errors (invalid authentication) to avoid wasting reconnection attempts.”
3. “How do you handle authentication securely in desktop plugins?”
Strong answer: “I never hardcode credentials. For Home Assistant, I use long-lived tokens stored in the plugin’s settings, which Stream Deck encrypts at rest. I validate tokens on save in the Property Inspector and provide clear error messages. For services requiring OAuth, I’d implement the authorization code flow with PKCE, opening the auth URL in the system browser and listening for the callback.”
4. “What happens if the user issues a command while the external service is unreachable?”
Strong answer: “I implement command queuing with timeout. When the user presses a key, I show a ‘pending’ state on the icon, attempt the WebSocket send, and start a timeout. If the connection is down, the send fails immediately and I show an error state. If the connection is up but the service doesn’t respond within the timeout, I revert to the previous state and show an error indicator. I also provide visual feedback that the system is reconnecting.”
Hints in Layers
Hint 1: Home Assistant WebSocket API Connection
Click to reveal
import WebSocket from 'ws';
class HomeAssistantConnection {
private ws: WebSocket | null = null;
private messageId = 0;
connect(url: string, token: string): Promise<void> {
return new Promise((resolve, reject) => {
this.ws = new WebSocket(`ws://${url}/api/websocket`);
this.ws.on('message', (data) => {
const message = JSON.parse(data.toString());
if (message.type === 'auth_required') {
// Send authentication
this.ws?.send(JSON.stringify({
type: 'auth',
access_token: token
}));
} else if (message.type === 'auth_ok') {
resolve();
} else if (message.type === 'auth_invalid') {
reject(new Error('Invalid authentication'));
}
});
});
}
}
Hint 2: Entity State Subscription
Click to reveal
// After authentication succeeds, subscribe to state changes
subscribeToStateChanges(): void {
const subscribeMessage = {
id: ++this.messageId,
type: 'subscribe_events',
event_type: 'state_changed'
};
this.ws?.send(JSON.stringify(subscribeMessage));
}
// Handle incoming state change events
handleMessage(message: any): void {
if (message.type === 'event' && message.event.event_type === 'state_changed') {
const { entity_id, new_state } = message.event.data;
// Find keys that control this entity and update their images
this.updateKeysForEntity(entity_id, new_state);
}
}
Hint 3: Bidirectional Sync Implementation
Click to reveal
// When user presses Stream Deck key
async onKeyDown(ev: KeyDownEvent): Promise<void> {
const entityId = ev.payload.settings.entityId;
// Optimistically update the key image
this.updateKeyImage(ev.action, 'pending');
try {
// Send command to Home Assistant
await this.callService(entityId, 'toggle');
// Don't update image here---wait for state_changed event
// This ensures the key reflects the actual device state
} catch (error) {
// Revert to previous state on failure
this.updateKeyImage(ev.action, 'error');
}
}
// Call a Home Assistant service
callService(entityId: string, service: string): Promise<void> {
const [domain] = entityId.split('.');
return this.sendCommand({
type: 'call_service',
domain,
service,
target: { entity_id: entityId }
});
}
Hint 4: Robust Error Recovery
Click to reveal
class RobustConnection {
private reconnectAttempts = 0;
private maxReconnectDelay = 30000;
private getReconnectDelay(): number {
// Exponential backoff: 1s, 2s, 4s, 8s... max 30s
const baseDelay = Math.min(
1000 * Math.pow(2, this.reconnectAttempts),
this.maxReconnectDelay
);
// Add jitter (plus or minus 20%)
const jitter = baseDelay * 0.2 * (Math.random() - 0.5);
return baseDelay + jitter;
}
private scheduleReconnect(): void {
const delay = this.getReconnectDelay();
this.reconnectAttempts++;
// Update all keys to show "reconnecting" state
this.updateAllKeysState('reconnecting');
setTimeout(() => {
this.connect()
.then(() => {
this.reconnectAttempts = 0;
this.resubscribeToStates();
})
.catch(() => this.scheduleReconnect());
}, delay);
}
private onDisconnect(): void {
if (this.wasCleanClose) return;
this.scheduleReconnect();
}
}
Books That Will Help
| Topic | Book | Chapter/Section | Why It Helps |
|---|---|---|---|
| State Synchronization | Designing Data-Intensive Applications | Ch. 5: Replication | Understand leader/follower patterns for keeping remote state in sync |
| Stream Processing | Designing Data-Intensive Applications | Ch. 11: Stream Processing | Event-driven architecture for real-time updates |
| WebSocket Patterns | Node.js Design Patterns | Ch. 3: Callbacks and Events | Managing async connections and event emitters |
| Error Recovery | Node.js Design Patterns | Ch. 11: Advanced Recipes | Reconnection and resilience patterns |
| TypeScript Error Handling | Effective TypeScript | Item 46: Use unknown | Safely handling WebSocket message types |
| API Authentication | Web Application Security | Ch. 4: Authentication | Token-based auth best practices |
| Reconnection Strategies | Release It! | Ch. 5: Stability Patterns | Circuit breakers and timeouts for resilient systems |
Project 4: Productivity Metrics Tracker
📚 Deep Dive: Complete Project Guide — Full theory, implementation guide, and exercises
- File: productivity_metrics_tracker_stream_deck.md
- Main Programming Language: TypeScript
- Alternative Programming Languages: JavaScript, Python, Rust
- Coolness Level: Level 2: Practical but Forgettable
- Business Potential: Level 2: The “Micro-SaaS / Pro Tool”
- Difficulty: Level 2: Intermediate (The Developer)
- Knowledge Area: Data Persistence, Visualization, Webhooks
- Software or Tool: Stream Deck, Notion, Google Sheets
- Main Book: Effective TypeScript by Dan Vanderkam
What you’ll build: A plugin that tracks how many times you press certain buttons (meeting started, task completed, coffee break), displays daily/weekly stats, and exports to a local CSV or sends to a webhook.
Why it teaches Stream Deck development: This project teaches data persistence, aggregation logic, data visualization on tiny screens, and external integrations. You’ll see your productivity patterns visualized on your desk.
Core challenges you’ll face:
- Local data persistence (maps to plugin data storage APIs)
- Time-based aggregation (maps to daily/weekly rollups)
- Mini-charts on key icons (maps to drawing bar/line charts in 72x72 pixels)
- Export mechanisms (maps to file I/O and HTTP webhooks)
Key Concepts:
- Plugin data storage: Settings Documentation - Elgato
- Date/time handling: date-fns - npm
- Tiny data visualization: Custom canvas rendering logic
- Webhook integrations: axios for HTTP calls
Difficulty: Intermediate Time estimate: 1-2 weeks Prerequisites: JavaScript/TypeScript, basic data structures
Real world outcome:
- “Meeting” key that increments your daily meeting count on each press
- Keys display mini bar charts showing the week’s activity
- Long-press exports your data to CSV in your Documents folder
- Optional webhook to send data to Notion, Google Sheets, or your own API
What your Stream Deck key looks like:
+------------------+
| Meetings: 4 |
| |
| ▂▄▆▃▇█▄ |
| M T W T F S S |
+------------------+
The mini bar chart shows this week's meeting counts:
- Monday: 2 meetings (▂)
- Tuesday: 4 meetings (▄)
- Wednesday: 6 meetings (▆)
- Thursday: 3 meetings (▃)
- Friday: 7 meetings (▇)
- Saturday: 8 meetings (█)
- Sunday: 4 meetings (▄)
CSV Export Output (~/Documents/productivity_metrics.csv):
+------------+----------+-------+--------+
| date | meetings | tasks | breaks |
+------------+----------+-------+--------+
| 2025-01-20 | 4 | 12 | 3 |
| 2025-01-21 | 6 | 8 | 4 |
| 2025-01-22 | 3 | 15 | 2 |
+------------+----------+-------+--------+
Learning milestones:
- First milestone: Counter action that persists across Stream Deck restarts
- Second milestone: Daily aggregation with reset at midnight
- Third milestone: Mini bar chart visualization on key
- Final milestone: CSV export and webhook integration via Property Inspector configuration
The Core Question You’re Answering
“How do I build persistent data storage with time-based aggregation and data visualization within the constraints of a Stream Deck plugin?”
This project teaches you that Stream Deck plugins can maintain meaningful state across days and weeks. You’re not just responding to button presses—you’re building a tiny database that accumulates data over time and presents it visually.
Concepts You Must Understand First
- Plugin Data Persistence - Stream Deck provides a data directory for each plugin. Where is it? How do you read/write files there?
- Book: Stream Deck SDK - Settings Guide
- Time-Based Aggregation - How do you track “today’s count” vs “this week’s counts”? What happens at midnight?
- Book: Effective TypeScript by Vanderkam - Item 28: Valid state representation
- Mini-Chart Rendering - Drawing a 7-bar chart in 72 pixels requires careful design. Each bar is ~8 pixels wide.
- Book: HTML5 Canvas by Fulton - Ch. 2: Drawing basics
- File I/O in Node.js - Synchronous vs asynchronous file operations, JSON serialization.
- Book: Node.js Design Patterns by Casciaro - Ch. 5: Async patterns
Questions to Guide Your Design
- Where does plugin data persist across restarts? (Hint:
streamDeck.plugin.dataPath) - How do you handle midnight rollover? (Timer? Check on each press?)
- How do you render a readable bar chart in 72 pixels? (Max 7-8 bars)
- What JSON structure stores daily counts efficiently?
- What format should the CSV export use?
Thinking Exercise
Design the data structure:
// Option A: Array of daily records
{
"days": [
{ "date": "2025-01-20", "meetings": 4, "tasks": 12, "breaks": 3 },
{ "date": "2025-01-21", "meetings": 6, "tasks": 8, "breaks": 4 }
]
}
// Option B: Nested by date
{
"2025-01-20": { "meetings": 4, "tasks": 12, "breaks": 3 },
"2025-01-21": { "meetings": 6, "tasks": 8, "breaks": 4 }
}
// Which is better for weekly aggregation?
// Which is better for CSV export?
The Interview Questions They’ll Ask
- “How do you persist data in a Stream Deck plugin that survives app restarts?”
- “Explain your approach to time-based data aggregation.”
- “How would you optimize storage if the user tracks data for years?”
- “How do you handle timezone issues with date-based aggregation?”
Hints in Layers
Hint 1: Use fs.readFileSync and fs.writeFileSync with streamDeck.plugin.dataPath for simple persistence.
Hint 2: Use date-fns library for reliable date manipulation: startOfDay(), format(), isSameDay().
Hint 3: For the bar chart, calculate max value first, then scale all bars relative to max.
Hint 4: For CSV, use Papa Parse or simple string concatenation with proper escaping.
Books That Will Help
| Topic | Book | Chapter |
|---|---|---|
| File I/O patterns | Node.js Design Patterns | Ch. 5: Async patterns |
| Type-safe state | Effective TypeScript | Item 28: Valid states |
| Canvas drawing | HTML5 Canvas by Fulton | Ch. 2-4 |
| Date handling | date-fns documentation | Formatting & comparison |
Project 5: Custom Soundboard with Waveform Display
📚 Deep Dive: Complete Project Guide — Full theory, implementation guide, and exercises
- File: ELGATO_STREAM_DECK_PLUGIN_LEARNING_PROJECTS.md
- Programming Language: JavaScript/TypeScript
- Coolness Level: Level 3: Genuinely Clever
- Business Potential: 2. The “Micro-SaaS / Pro Tool”
- Difficulty: Level 3: Advanced
- Knowledge Area: Audio Processing / Graphics
- Software or Tool: Web Audio API / Canvas
- Main Book: “HTML5 Canvas” by Steve Fulton
What you’ll build: A soundboard plugin where each key plays a configurable audio file and shows an animated waveform visualization on the key while playing.
Why it teaches Stream Deck development: This project combines file handling, audio playback, real-time visualization, and multi-platform considerations. It’s technically challenging and produces a genuinely fun result.
Core challenges you’ll face:
- Audio playback in Node.js (maps to platform-specific audio APIs)
- Real-time waveform generation (maps to FFT analysis and canvas drawing)
- File picker in Property Inspector (maps to Electron-style file dialogs)
- Performance optimization (maps to efficient rendering at 60fps equivalent)
Resources for key challenges:
- node-wav - WAV file parsing
- web-audio-api - Audio analysis in Node.js
Key Concepts:
- Audio in Node.js: play-sound - npm
- FFT for waveforms: Web Audio API concepts
- High-frequency image updates: Optimization techniques for Stream Deck
- Cross-platform file paths: Node.js
pathmodule best practices
Difficulty: Advanced Time estimate: 2-3 weeks Prerequisites: Strong JavaScript/TypeScript, understanding of audio concepts, canvas drawing
Real world outcome:
- Press a key, hear your sound, see the waveform animate on the LCD
- Configure any MP3/WAV file through the Property Inspector
- Visual feedback shows playback progress
- A genuinely impressive demo that showcases Stream Deck’s capabilities
Visual progression of your soundboard key:
IDLE STATE PLAYING STATE FINISHED STATE
+-------------+ +-------------+ +-------------+
| | | | | | | |
| [PLAY] | | || ||| | | | [REPLAY] |
| icon | |||||||||||||| | icon |
| | |||||||||||||| | |
| drum.wav | | ^^^^^^^^^^ | | drum.wav |
+-------------+ +-------------+ +-------------+
Animated bars move
with audio frequency!
Property Inspector file picker:
+------------------------------------------+
| SOUNDBOARD CONFIGURATION |
+------------------------------------------+
| |
| Audio File: |
| +----------------------------------+ |
| | /Users/me/sounds/airhorn.mp3 | |
| +----------------------------------+ |
| [ Browse... ] |
| |
| Volume: [=======------] 70% |
| |
| Waveform Style: |
| ( ) Bars (*) Wave ( ) Circle |
| |
| Color: [#00FF88] (color picker) |
| |
+------------------------------------------+
Learning milestones:
- First milestone: Key press plays a hardcoded sound file
- Second milestone: Property Inspector file selector for custom sounds
- Third milestone: Static waveform visualization generated from audio
- Final milestone: Animated real-time waveform during playback
The Core Question You’re Answering
How do I integrate audio playback with real-time visual feedback, processing audio data for animated waveform display on a 72x72 pixel key?
This project forces you to solve a multi-domain problem: audio processing (a backend concern), real-time rendering (a graphics concern), and efficient communication (a systems concern). The Stream Deck’s small display and WebSocket-based image updates create unique constraints that make this genuinely challenging.
Concepts You Must Understand First
Before writing any code, you need to understand these foundational concepts:
1. Audio Playback in Node.js (Platform-Specific Approaches)
Unlike the browser where <audio> elements “just work,” Node.js has no built-in audio playback. You must use external libraries that wrap platform-specific audio systems:
- macOS: Uses
afplaycommand or Core Audio APIs - Windows: Uses Windows Media Player COM objects or DirectSound
- Linux: Uses ALSA or PulseAudio
Reference: “Node.js Design Patterns” by Mario Casciaro - Chapter on external process integration
2. FFT (Fast Fourier Transform) for Waveform Analysis
FFT converts audio from the time domain (amplitude over time) to the frequency domain (which frequencies are present). This is how you get those classic “visualizer bars” that react to bass, mids, and treble:
Time Domain (raw audio): Frequency Domain (after FFT):
^ ^
| /\ /\ | |||
| / \ / \ | |||||
|/ \/ \ ||||||||||
+-------------> +------------->
time frequency
Reference: “Digital Signal Processing” by Steven W. Smith - Chapters 8-12 on FFT
3. High-Frequency Canvas Rendering (Animation Frames)
You need to generate images fast enough to appear animated (15-30 FPS), but the Stream Deck’s WebSocket can become a bottleneck. Key considerations:
- Canvas operations are synchronous and can block the event loop
- Base64 encoding images adds CPU overhead
- WebSocket message queuing can cause lag if you send too fast
Reference: “HTML5 Canvas” by Steve Fulton - Chapter 5 on animation loops
4. File Picker Integration in Property Inspector
The Property Inspector runs in a web context, so you cannot directly access the filesystem. You must use HTML5 <input type="file"> and communicate the selected path to the plugin backend.
Reference: Stream Deck SDK documentation on Property Inspector communication
5. Cross-Platform File Path Handling
Windows uses backslashes (C:\Users\...), macOS/Linux use forward slashes (/Users/...). The Node.js path module normalizes this, but you must be careful when storing paths in settings.
Reference: “Node.js in Action” by Mike Cantelon - Chapter on filesystem operations
Questions to Guide Your Design
Before implementing, think through these questions:
- How do you play audio from Node.js (not browser)?
- The plugin runs in Node.js, not a browser. You cannot use
<audio>elements or Web Audio API directly. - What library handles cross-platform audio playback?
- How do you know when playback finishes?
- The plugin runs in Node.js, not a browser. You cannot use
- How do you extract waveform data from an audio file?
- Do you analyze in real-time during playback, or pre-compute the waveform?
- If pre-computing, how do you synchronize the visualization with playback position?
- What format does FFT output, and how do you map it to bar heights?
- How do you animate the key display without overloading the WebSocket?
- What frame rate is “good enough” for a 72x72 display?
- How do you throttle updates without making animation jerky?
- Should you batch multiple frame updates?
- How do you handle different audio formats (MP3, WAV, OGG)?
- Which formats can you decode in Node.js?
- Do you need external codecs?
- How do you handle unsupported formats gracefully?
Thinking Exercise
Trace the complete audio playback flow from file selection to animated display:
Step 1: File Selected in Property Inspector
+------------------------------------------+
| User clicks "Browse..." and selects |
| "airhorn.mp3" from their computer |
+------------------------------------------+
|
v
Step 2: Path Sent to Plugin Backend
+------------------------------------------+
| Property Inspector sends message: |
| { action: "setAudioFile", |
| payload: "/Users/me/airhorn.mp3" } |
+------------------------------------------+
|
v
Step 3: Audio File Loaded and Analyzed
+------------------------------------------+
| Plugin reads file, decodes audio, |
| runs FFT to pre-compute waveform data |
| (array of frequency magnitudes) |
+------------------------------------------+
|
v
Step 4: User Presses Key - Playback Starts
+------------------------------------------+
| onKeyDown fires, plugin calls |
| play-sound library to start audio |
+------------------------------------------+
|
v
Step 5: Animation Loop Begins
+------------------------------------------+
| setInterval at 30fps: |
| - Calculate current playback position |
| - Get FFT data for current position |
| - Draw bars on Canvas |
| - Convert to base64 |
| - Call setImage() via WebSocket |
+------------------------------------------+
|
v
Step 6: Display Updates on Stream Deck
+------------------------------------------+
| Each frame: new image appears on key |
| Bars animate in sync with audio! |
+------------------------------------------+
|
v
Step 7: Playback Finishes
+------------------------------------------+
| Audio ends, stop animation loop, |
| display "REPLAY" icon or reset |
+------------------------------------------+
Question to ponder: At Step 5, why might pre-computed FFT data be better than real-time analysis? What are the tradeoffs?
The Interview Questions They’ll Ask
If you can answer these, you truly understand the project:
- “How does FFT work for audio visualization?”
- Explain the time-to-frequency domain transformation
- Describe how FFT bin indices map to frequency ranges
- Discuss why you might use logarithmic scaling for visual appeal
- “What’s your approach to high-frequency image updates?”
- Discuss frame rate selection (15-30 FPS for Stream Deck)
- Explain throttling mechanisms (setTimeout vs setInterval vs requestAnimationFrame equivalent)
- Describe how you avoid WebSocket message queue buildup
- “How do you handle cross-platform audio playback?”
- Explain why Node.js lacks built-in audio
- Compare libraries: play-sound, node-speaker, node-audio-player
- Discuss platform detection and fallback strategies
- “Why did you choose pre-computed vs real-time waveform analysis?”
- Pre-computed: Lower CPU during playback, perfect sync, upfront processing time
- Real-time: Lower memory, works with streaming, more complex implementation
Hints in Layers
Try to solve each challenge yourself first. Only reveal hints if stuck:
Hint 1: Which library plays audio in Node.js?
The play-sound library is the simplest cross-platform solution. It spawns platform-specific commands:
const player = require('play-sound')();
player.play('/path/to/sound.mp3', (err) => {
if (err) console.error('Playback failed:', err);
else console.log('Playback finished');
});
For more control (pause, seek, volume), consider node-audio-player or howler.js (if you can bundle a browser context).
Hint 2: Pre-computed waveform vs real-time FFT?
Pre-computing is recommended for Stream Deck because:
- You can analyze the entire file once during setup
- During playback, you just index into the pre-computed array based on time
- No CPU-intensive FFT during the animation loop
Use a library like node-wav to decode audio, then run FFT on chunks:
// Pseudo-code for pre-computing waveform
const samples = decodeAudioFile('/path/to/file.wav');
const chunkSize = sampleRate / 30; // 30 chunks per second
const waveformData = [];
for (let i = 0; i < samples.length; i += chunkSize) {
const chunk = samples.slice(i, i + chunkSize);
const fftResult = performFFT(chunk);
waveformData.push(fftResult);
}
Hint 3: Frame rate limiting for setImage
Do not use setInterval with exact frame timing - WebSocket latency can cause buildup. Instead, use a “frame complete” pattern:
let isRendering = false;
const TARGET_FPS = 25;
const FRAME_TIME = 1000 / TARGET_FPS;
async function renderLoop() {
if (isRendering) return;
isRendering = true;
const frameStart = Date.now();
// Draw frame
const imageData = drawWaveformFrame(currentPosition);
await action.setImage(imageData);
// Schedule next frame, accounting for render time
const elapsed = Date.now() - frameStart;
const delay = Math.max(0, FRAME_TIME - elapsed);
setTimeout(() => {
isRendering = false;
if (isPlaying) renderLoop();
}, delay);
}
Hint 4: File picker in Property Inspector HTML
The Property Inspector uses standard HTML, but file paths need special handling:
<!-- In Property Inspector HTML -->
<input type="file" id="audioFile" accept=".mp3,.wav,.ogg" />
<script>
document.getElementById('audioFile').addEventListener('change', (e) => {
const file = e.target.files[0];
if (file) {
// Note: file.path may not be available in all contexts
// You might need to read the file and send data, not path
sendToPlugin({
action: 'setAudioFile',
filename: file.name,
path: file.path // Electron-style, may not work everywhere
});
}
});
</script>
For cross-platform reliability, consider reading the file as ArrayBuffer and sending the audio data itself, then saving it to the plugin’s data directory.
Books That Will Help
| Book | Author | Why It Helps |
|---|---|---|
| “HTML5 Canvas” | Steve Fulton, Jeff Fulton | Definitive guide to canvas drawing, animation loops, and optimization |
| “Digital Signal Processing” | Steven W. Smith | Free online, explains FFT and audio analysis from first principles |
| “Node.js Design Patterns” | Mario Casciaro | Patterns for async operations, streams, and external process integration |
| “Web Audio API” | Boris Smus | While browser-focused, explains audio concepts that apply to Node.js |
| “Node.js in Action” | Mike Cantelon et al. | Comprehensive guide including filesystem and cross-platform concerns |
Project Comparison Table
| Project | Difficulty | Time | Depth of Understanding | Fun Factor |
|---|---|---|---|---|
| Pomodoro Timer | Beginner | Weekend | ⭐⭐⭐ Core concepts | ⭐⭐⭐⭐ Useful daily |
| System Monitor | Beginner-Intermediate | 1 week | ⭐⭐⭐⭐ Multi-action | ⭐⭐⭐⭐ Genuinely useful |
| Smart Home Controller | Intermediate | 1-2 weeks | ⭐⭐⭐⭐⭐ External APIs | ⭐⭐⭐⭐⭐ Mind-blowing |
| Productivity Tracker | Intermediate | 1-2 weeks | ⭐⭐⭐⭐ Data & viz | ⭐⭐⭐ Niche appeal |
| Waveform Soundboard | Advanced | 2-3 weeks | ⭐⭐⭐⭐⭐ Performance | ⭐⭐⭐⭐⭐ Very impressive |
Recommendation
Based on the progression of skills, I recommend starting with:
Start Here → Project 1: Pomodoro Timer
This teaches you the complete plugin development workflow without external complexity. You’ll learn:
- CLI scaffolding (
streamdeck create) - Manifest configuration
- Action classes and event handling
- Dynamic image updates
- Property Inspector basics
- Settings persistence
Once completed, move to Project 2: System Monitor to learn multi-action plugins, then Project 3: Smart Home Controller for the full external integration experience.
Final Capstone Project: Stream Deck Macro Automation Suite
- File: ELGATO_STREAM_DECK_PLUGIN_LEARNING_PROJECTS.md
- Main Programming Language: TypeScript
- Alternative Programming Languages: JavaScript, Rust (via wasm-bindgen)
- Coolness Level: Level 5: Absolutely Mind-Blowing
- Business Potential: Level 3: The “Actual Product”
- Difficulty: Level 4: Expert (The Architect)
- Knowledge Area: System Automation / Plugin Architecture / Cross-Platform Development
- Software or Tool: Stream Deck SDK, Node.js child_process, AppleScript/PowerShell
- Main Book: “Node.js Design Patterns” by Mario Casciaro & Luciano Mammino
What you’ll build: A comprehensive automation plugin that combines keyboard shortcuts, application launching, API triggers, conditional logic, and multi-step sequences—essentially building your own simplified version of Keyboard Maestro or AutoHotkey, controlled entirely from Stream Deck.
Why it teaches everything: This project synthesizes all prior learning into a professional-grade plugin:
- Complex multi-action architecture
- Conditional execution (if app X is focused, do Y)
- Sequence/macro recording and playback
- Cross-platform system integration (AppleScript on macOS, PowerShell on Windows)
- Robust error handling and logging
- Plugin distribution and packaging
Core challenges you’ll face:
- Cross-platform automation (maps to platform-specific scripting)
- Macro recording/playback engine (maps to action sequence storage and execution)
- Conditional triggers (maps to system state detection)
- Complex Property Inspector UI (maps to building a mini-IDE for macro editing)
- Plugin packaging for distribution (maps to
streamdeck packand Elgato marketplace)
Resources for key challenges:
- Stream Deck CLI & Pack Command - Official packaging workflow
- active-win npm package - Cross-platform window detection
- Sortable.js - Drag-and-drop for macro step reordering
Key Concepts:
- System automation: AppleScript, PowerShell, Node.js child processes
- Sequence execution engines: State machines and async flow control
- Complex UI in Property Inspector: Modern HTML/CSS frameworks in constrained environments
- Plugin distribution: Stream Deck CLI pack command - Elgato
Difficulty: Advanced-Expert Time estimate: 1 month+ Prerequisites: All prior projects (especially Projects 1-3), strong JavaScript/TypeScript, platform scripting knowledge (AppleScript or PowerShell basics)
Real World Outcome
You’ll have built a complete automation platform—think Keyboard Maestro or AutoHotkey, but controlled from your Stream Deck. Press a key, and complex multi-step workflows execute automatically with visual feedback on the key itself.
Plugin Running - Console Output:
$ npm run watch
[MACRO] Plugin started - Macro Automation Suite v1.0.0
[MACRO] Registered actions:
├── com.yourname.macro.execute (Run saved macro)
├── com.yourname.macro.record (Record new macro)
└── com.yourname.macro.conditional (Conditional action)
[MACRO] Loaded 12 saved macros from data directory
[MACRO] WebSocket connected to Stream Deck
[INFO] User pressed "Quick Standup" key
[EXEC] Starting macro: "Quick Engineering Standup" (7 steps)
[STEP 1/7] ▶ Launch App: Slack
├── Platform: darwin
├── Command: open -a "Slack"
└── Result: ✓ Success (Slack focused)
[STEP 2/7] ▶ Keystroke: Cmd+K
├── Sending: osascript -e 'tell application "System Events" to keystroke "k" using command down'
└── Result: ✓ Success
[STEP 3/7] ▶ Type Text: "engineering"
├── Characters: 11
└── Result: ✓ Success
[STEP 4/7] ▶ Wait: 500ms
└── Result: ✓ Complete
[STEP 5/7] ▶ Keystroke: Enter
└── Result: ✓ Success
[STEP 6/7] ▶ Conditional: Clipboard not empty
├── Condition: clipboard.length > 0
├── Evaluated: true (clipboard has 247 chars)
└── Executing nested action: Paste
[STEP 6a] ▶ Keystroke: Cmd+V
└── Result: ✓ Success
[STEP 7/7] ▶ Keystroke: Enter (send message)
└── Result: ✓ Success
[EXEC] ✓ Macro completed successfully in 2.3s
[IMAGE] Updating key to "Sent!" confirmation state
[IMAGE] Will revert to default icon in 2000ms
Conditional Macro Execution:
[INFO] User pressed "Smart Zoom Toggle" key
[COND] Evaluating conditions for macro: "Smart Zoom Toggle"
├── Condition 1: Is "zoom.us" running?
│ └── Checking active processes...
│ └── Result: TRUE (PID 12847)
├── Condition 2: Is meeting active?
│ └── Checking window title for "Meeting"...
│ └── Result: TRUE (window: "Engineering Standup - Zoom")
└── All conditions: TRUE → Executing "mute_action"
[EXEC] Starting macro: "Toggle Mute" (1 step)
[STEP 1/1] ▶ Keystroke: Cmd+Shift+A (Zoom mute toggle)
└── Result: ✓ Success
[EXEC] ✓ Macro completed - microphone muted
[IMAGE] Updating key: 🔇 (muted indicator)
A Complete Macro Example - “Quick Engineering Standup”:
Macro: "Quick Engineering Standup"
Trigger: Stream Deck key press
Steps:
1. Open Slack (or focus if already running)
2. Navigate to #engineering channel (Cmd+K, type "engineering", Enter)
3. Wait 500ms for channel to load
4. Paste clipboard contents (today's standup update)
5. If "send_immediately" setting is true: Press Enter to send
6. Play confirmation sound
7. Show "Sent!" on Stream Deck key for 2 seconds
Stream Deck Key Visual States:
┌─────────────────────────────────────────────────────────────────────────┐
│ MACRO KEY VISUAL STATES │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ IDLE STATE RUNNING STATE SUCCESS STATE ERROR STATE │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ │ │ Step 3/7 │ │ │ │ │ │
│ │ [MACRO] │ ──► │ ████░░░░ │ ──► │ ✓ ✓ │ or │ ✗ │ │
│ │ Standup │ │ Typing...│ │ SENT! │ │ FAILED │ │
│ │ │ │ │ │ │ │ │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
│ (Blue bg) (Animated) (Green, 2s) (Red, 3s) │
│ │
│ Key shows real-time progress as macro executes! │
│ ───────────────────────────────────────────────── │
│ Progress bar fills as steps complete │
│ Current step name displayed on key │
│ Color indicates state (blue=idle, yellow=running, green=done) │
│ │
└─────────────────────────────────────────────────────────────────────────┘
Conditional Workflow Examples:
- “If Zoom is running AND meeting is active, mute mic; otherwise, open Zoom and join scheduled meeting”
- “If current time is after 6pm, send ‘Signing off!’ to Slack and close work apps”
- “If clipboard contains a URL, open it in browser; if it’s text, create a new note”
- “If VS Code is focused, run current file; if Terminal is focused, clear and run last command”
The Property Inspector becomes a mini-IDE for macro creation:
+-------------------------------------------------------------------------+
| MACRO EDITOR - Property Inspector |
+-------------------------------------------------------------------------+
| Macro Name: [Quick Engineering Standup ] |
| |
| +---------------------------------------------------------------------+ |
| | STEPS [+ Add Step] | |
| | ------------------------------------------------------------------ | |
| | +---------------------------------------------------------------+ | |
| | | 1. [Launch App v] App: [Slack v] [x] | | |
| | +---------------------------------------------------------------+ | |
| | +---------------------------------------------------------------+ | |
| | | 2. [Keystroke v] Keys: [Cmd+K ] [x] | | |
| | +---------------------------------------------------------------+ | |
| | +---------------------------------------------------------------+ | |
| | | 3. [Type Text v] Text: [engineering ] [x] | | |
| | +---------------------------------------------------------------+ | |
| | +---------------------------------------------------------------+ | |
| | | 4. [Wait v] Duration: [500 ] ms [x] | | |
| | +---------------------------------------------------------------+ | |
| | +---------------------------------------------------------------+ | |
| | | 5. [Conditional v] IF: [Clipboard not empty ] [x] | | |
| | | +-- [Paste v] | | |
| | +---------------------------------------------------------------+ | |
| +---------------------------------------------------------------------+ |
| |
| +-- CONDITIONS -------------------------------------------------------+ |
| | [x] Only run if: [Slack is running v] | |
| | [ ] Time restriction: [ ] After [ ] Before [ ] | |
| +---------------------------------------------------------------------+ |
| |
| [Test Macro] [Export JSON] [Import] [Save Macro] |
+-------------------------------------------------------------------------+
Your plugin in the Stream Deck Store (concept):
+-------------------------------------------------------------------------+
| STREAM DECK STORE [Search...] |
+-------------------------------------------------------------------------+
| |
| +-------------------------------------------------------------------+ |
| | [ICON] MACRO AUTOMATION SUITE | |
| | by Your Name | |
| | ***** (127 reviews) | |
| | | |
| | Build powerful multi-step automations with conditional logic, | |
| | cross-platform support, and a visual macro editor. | |
| | | |
| | [Install] [View Details] | |
| +-------------------------------------------------------------------+ |
| |
+-------------------------------------------------------------------------+
Import/export macro configurations for backup and sharing between machines
Learning milestones:
- First milestone: Single-step automation (launch app, send keystroke)
- Second milestone: Multi-step sequences with delays and error handling
- Third milestone: Visual macro editor in Property Inspector
- Fourth milestone: Conditional triggers based on active application
- Final milestone: Package and distribute as a complete plugin
The Core Question You’re Answering
“How do I build a complete automation platform with cross-platform scripting, conditional logic, sequence recording, and a complex Property Inspector UI that could be published to the Elgato Marketplace?”
This capstone synthesizes everything: manifest configuration, WebSocket communication, settings persistence, dynamic rendering, external system integration, and building something polished enough for distribution.
Concepts You Must Understand First
Before writing any code, you need deep understanding of these foundational concepts. This is an expert-level project that synthesizes everything you’ve learned:
| Concept | Why It Matters | Where to Learn |
|---|---|---|
| Cross-Platform Automation | macOS uses AppleScript, Windows uses PowerShell. You need an abstraction layer (Adapter pattern) that lets macro steps work identically on both platforms. | Node.js Design Patterns Ch. 8: Structural Patterns (Adapter pattern) |
| Child Process Management | Every automation step spawns a system process. You must handle stdout/stderr, exit codes, timeouts, and zombie processes. | Node.js Design Patterns Ch. 9: Behavioral Patterns; Node.js child_process docs |
| Sequence Execution Engine | Running steps in order with delays, conditionals, error handling, and rollback requires a robust state machine. | Enterprise Integration Patterns by Hohpe - Process Manager pattern |
| Window/Application Detection | Conditionals need to detect active app, window title, running processes. Cross-platform APIs differ significantly. | active-win npm package; macOS Accessibility APIs; Windows UI Automation |
| Async Flow Control | Steps are async (child processes, timeouts). You need proper Promise chains or async/await with error boundaries. | Node.js Design Patterns Ch. 5: Asynchronous Control Flow Patterns |
| Complex Property Inspector UI | Building a drag-and-drop macro editor in HTML/CSS/JS that communicates with your plugin backend. | Sortable.js docs; Stream Deck PI best practices |
| JSON Schema Design | Your macro format is a mini-DSL. Schema design determines extensibility, validation, and import/export compatibility. | Designing Data-Intensive Applications Ch. 4: Encoding and Evolution |
| Plugin Packaging & Distribution | streamdeck pack, code signing, Elgato marketplace submission process, and handling updates. |
Stream Deck SDK CLI docs; Elgato Developer Program |
Deep Dive: The Adapter Pattern for Cross-Platform Automation
The core architectural challenge is making this code work identically on macOS and Windows:
┌─────────────────────────────────────────────────────────────────────────────┐
│ CROSS-PLATFORM AUTOMATION ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ Macro Step Automation Adapter Platform │
│ ────────── ────────────────── ────────── │
│ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌──────────────┐ │
│ │ { type: "launch"│ │ launchApp() │ │ macOS: │ │
│ │ app: "Slack" }│ ──► │ │ ──► │ open -a │ │
│ └─────────────────┘ │ ┌───────────┐ │ │ "Slack" │ │
│ │ │ Platform │ │ └──────────────┘ │
│ ┌─────────────────┐ │ │ Detector │ │ │
│ │ { type: "key" │ ──► │ └───────────┘ │ ──► ┌──────────────┐ │
│ │ keys: "cmd+k"}│ │ │ │ │ Windows: │ │
│ └─────────────────┘ │ ▼ │ │ Start-Process│ │
│ │ ┌───────────┐ │ │ slack.exe │ │
│ ┌─────────────────┐ │ │ macOS │ │ └──────────────┘ │
│ │ { type: "type" │ ──► │ │ Adapter │ │ │
│ │ text: "hello"}│ │ └───────────┘ │ │
│ └─────────────────┘ │ │ │ │
│ │ ┌───────────┐ │ │
│ │ │ Windows │ │ │
│ │ │ Adapter │ │ │
│ │ └───────────┘ │ │
│ └─────────────────┘ │
│ │
│ The Adapter Pattern: Same interface, platform-specific implementations │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Deep Dive: The Sequence Execution Engine
Your macro runner is essentially a tiny workflow engine:
┌─────────────────────────────────────────────────────────────────────────────┐
│ MACRO EXECUTION STATE MACHINE │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌───────────────┐ │
│ │ IDLE │ │
│ │ (waiting for │ │
│ │ key press) │ │
│ └───────┬───────┘ │
│ │ KEY_DOWN event │
│ ▼ │
│ ┌───────────────┐ │
│ │ EVALUATING │ │
│ │ CONDITIONS │──────── Conditions FALSE ────┐ │
│ └───────┬───────┘ │ │
│ │ Conditions TRUE │ │
│ ▼ │ │
│ ┌───────────────┐ │ │
│ ┌───────►│ EXECUTING │ │ │
│ │ │ STEP N │ │ │
│ │ └───────┬───────┘ │ │
│ │ │ │ │
│ │ ┌──────────┼──────────┐ │ │
│ │ │ │ │ │ │
│ │ ▼ ▼ ▼ │ │
│ │ SUCCESS TIMEOUT ERROR │ │
│ │ │ │ │ │ │
│ │ │ ▼ ▼ │ │
│ │ │ ┌───────────────────┐ │ │
│ │ │ │ ERROR HANDLING │ │ │
│ │ │ │ (retry? skip? │ │ │
│ │ │ │ abort? rollback)│ │ │
│ │ │ └─────────┬─────────┘ │ │
│ │ │ │ │ │
│ │ ▼ │ │ │
│ │ More steps? │ │ │
│ │ │ │ │ │
│ │ YES│ NO │ │ │
│ └─────┘ ▼ │ │
│ ┌───────────────┐ │ │
│ │ COMPLETE │◄───────────────────────┘ │
│ │ (update key │ │
│ │ show result)│ │
│ └───────┬───────┘ │
│ │ timeout or next press │
│ ▼ │
│ ┌───────────────┐ │
│ │ IDLE │ │
│ └───────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Questions to Guide Your Design
Work through these questions before coding. They’ll surface the architectural decisions that determine success or failure:
Cross-Platform Architecture
- How do you abstract platform-specific automation (AppleScript vs PowerShell)?
- Should you use a factory pattern, adapter pattern, or strategy pattern?
- How do you handle features that exist on one platform but not another?
- What’s your fallback when a platform-specific operation fails?
- How do you handle keyboard shortcuts that differ between platforms?
Cmd+Con macOS vsCtrl+Con Windows—who translates?- What about keys that don’t exist on both platforms (e.g., Windows key)?
- How do you represent modifier keys in your macro JSON schema?
Macro Execution Engine
- How do you serialize/deserialize macro sequences to JSON?
- What’s your schema for representing steps, conditionals, and loops?
- How do you handle versioning when you add new step types?
- What validation do you perform on import?
- How do you handle step timing and synchronization?
- What if a “launch app” step takes 5 seconds? Do you wait?
- How do you detect when an application is “ready” after launching?
- What’s your timeout strategy for each step type?
- How do you implement conditionals and branching?
- If/else? Switch/case? Nested conditionals?
- How deep can conditionals nest before the UI becomes unusable?
- How do you represent the condition logic in JSON?
Error Handling & Recovery
- What happens when a step fails mid-macro?
- Stop immediately? Skip and continue? Retry N times?
- Should there be an “undo” or “rollback” mechanism?
- How do you report which step failed and why?
- How do you handle the “app not installed” case?
- User configures a macro for Slack, but Slack isn’t installed
- Detect at macro save time? At runtime? Both?
Property Inspector UI
- How do you build a visual macro editor in HTML?
- How does drag-and-drop reordering work with Sortable.js?
- How do you communicate step changes to the plugin backend?
- How do you validate the macro before saving?
- How do you handle the import/export workflow?
- JSON download? Clipboard? Both?
- What about sharing macros between users?
System Integration
- How do you detect the currently focused application cross-platform?
- What npm packages help? (
active-win,frontmost-app) - How often do you poll for focus changes?
- What’s the performance impact of frequent polling?
- What npm packages help? (
Distribution
- What’s the
streamdeck packprocess for distribution?- What files must be included in the bundle?
- How do you handle native dependencies (node-gyp)?
- What’s the Elgato marketplace submission process?
Thinking Exercise
Before coding, design the complete macro execution flow on paper:
Part 1: Design the Macro JSON Schema
Start with this basic macro and extend it:
{
"name": "Open Slack Engineering",
"version": 1,
"steps": [
{ "type": "launch", "app": "Slack" },
{ "type": "delay", "ms": 2000 },
{ "type": "keystroke", "keys": "cmd+k" },
{ "type": "type", "text": "engineering" },
{ "type": "keystroke", "keys": "enter" }
]
}
Now extend this schema to support:
- Conditionals:
{ "type": "if", "condition": {...}, "then": [...], "else": [...] } - Loops:
{ "type": "repeat", "times": 3, "steps": [...] } - Variables:
{ "type": "setVariable", "name": "count", "value": 0 } - Error handling:
{ "type": "try", "steps": [...], "catch": [...] }
Draw your extended schema on paper before implementing.
Part 2: Trace the Execution Flow
Given this conditional macro:
{
"name": "Smart Meeting Toggle",
"conditions": [
{ "type": "appRunning", "app": "zoom.us" }
],
"ifTrue": [
{ "type": "keystroke", "keys": "cmd+shift+a" }
],
"ifFalse": [
{ "type": "launch", "app": "zoom.us" },
{ "type": "delay", "ms": 3000 },
{ "type": "keystroke", "keys": "cmd+j" }
]
}
Trace through these scenarios:
SCENARIO A: Zoom is running
─────────────────────────────────────────────────────────────
T=0ms: KEY_DOWN event received
→ What state does the engine enter? _______________
T=5ms: Evaluate condition "appRunning: zoom.us"
→ How do you check if Zoom is running? _______________
→ Result: TRUE
T=10ms: Execute ifTrue[0]: keystroke "cmd+shift+a"
→ What system command do you run? _______________
T=50ms: Step complete, no more steps
→ What state do you enter? _______________
T=50ms: Update key image
→ What do you display? _______________
SCENARIO B: Zoom is NOT running
─────────────────────────────────────────────────────────────
T=0ms: KEY_DOWN event received
T=5ms: Evaluate condition: FALSE
T=10ms: Execute ifFalse[0]: launch "zoom.us"
→ What command launches Zoom on macOS? _______________
→ What about Windows? _______________
T=3010ms: Execute ifFalse[1]: delay 3000ms
→ Why do we wait after launching? _______________
T=3020ms: Execute ifFalse[2]: keystroke "cmd+j"
→ What does Cmd+J do in Zoom? _______________
Part 3: Design the Error Recovery Matrix
Fill in this matrix for each step type:
┌────────────────┬─────────────────┬────────────────┬─────────────────┐
│ Step Type │ Possible Errors │ Default Action │ User Config? │
├────────────────┼─────────────────┼────────────────┼─────────────────┤
│ launch │ App not found │ ? │ ? │
│ │ Launch timeout │ ? │ ? │
├────────────────┼─────────────────┼────────────────┼─────────────────┤
│ keystroke │ No focused app │ ? │ ? │
│ │ Key not sent │ ? │ ? │
├────────────────┼─────────────────┼────────────────┼─────────────────┤
│ type │ Text too long │ ? │ ? │
│ │ Special chars │ ? │ ? │
├────────────────┼─────────────────┼────────────────┼─────────────────┤
│ delay │ Interrupted │ ? │ ? │
├────────────────┼─────────────────┼────────────────┼─────────────────┤
│ conditional │ Condition error │ ? │ ? │
└────────────────┴─────────────────┴────────────────┴─────────────────┘
Part 4: Architect the Plugin Structure
Draw the file/class structure:
com.yourname.macrosuite.sdPlugin/
├── manifest.json ← What actions do you define?
├── src/
│ ├── plugin.ts ← Main entry point
│ ├── actions/
│ │ ├── ExecuteAction.ts ← Runs a saved macro
│ │ ├── RecordAction.ts ← Records new macro
│ │ └── ConditionalAction.ts ← Smart conditional
│ ├── engine/
│ │ ├── MacroRunner.ts ← The execution engine
│ │ ├── StepExecutor.ts ← Runs individual steps
│ │ └── ConditionEvaluator.ts ← Evaluates conditions
│ ├── automation/
│ │ ├── AutomationAdapter.ts ← Abstract interface
│ │ ├── MacOSAdapter.ts ← AppleScript commands
│ │ └── WindowsAdapter.ts ← PowerShell commands
│ └── storage/
│ └── MacroStore.ts ← Save/load macros
├── ui/
│ ├── macro-editor.html ← Property Inspector
│ ├── js/
│ │ ├── editor.js ← Macro editor logic
│ │ └── sortable.min.js ← Drag-and-drop
│ └── css/
│ └── editor.css ← Custom styles
└── assets/
└── icons/ ← Action icons
Questions:
- Why separate
MacroRunnerfromStepExecutor? - Why is
AutomationAdapterabstract? - What’s the communication path from Property Inspector to MacroStore?
The Interview Questions They’ll Ask
After building this project, you should be able to answer these with confidence:
1. “How do you handle cross-platform compatibility for system automation?”
Strong answer: “I use the Adapter pattern to create a unified interface for automation operations. The AutomationAdapter interface defines methods like launchApp(), sendKeystroke(), and typeText(). Then I have platform-specific implementations: MacOSAdapter uses AppleScript via osascript, while WindowsAdapter uses PowerShell via child_process.exec(). At runtime, I detect process.platform and instantiate the appropriate adapter. This way, the macro execution engine is completely platform-agnostic—it just calls adapter.launchApp('Slack') and the adapter handles the translation.”
2. “Describe your macro execution engine architecture.”
Strong answer: “The engine is a state machine with states: IDLE, EVALUATING, EXECUTING, COMPLETE, and ERROR. When a key is pressed, it transitions from IDLE to EVALUATING, where it checks any pre-conditions. If conditions pass, it moves to EXECUTING and iterates through steps sequentially. Each step is handled by a StepExecutor that returns a Promise. I use async/await with try/catch to handle errors. The engine emits events for progress updates, which the action uses to update the key image in real-time. On completion or error, it transitions to COMPLETE, updates the key, and schedules a return to IDLE after a timeout.”
3. “How would you implement conditional actions based on system state?”
Strong answer: “Conditions are evaluated by a ConditionEvaluator class that supports multiple condition types. For ‘app running’ checks, I use the active-win npm package or shell out to pgrep on macOS / tasklist on Windows. For ‘time of day’ conditions, I use JavaScript’s Date object. For ‘clipboard contains’ conditions, I use clipboardy. Each condition type has a method that returns a boolean Promise. Complex conditions are composed using AND/OR operators, and the evaluator short-circuits when possible. Conditions are evaluated just before execution, not at macro save time, so they reflect current system state.”
4. “Walk me through packaging a plugin for marketplace distribution.”
Strong answer: “First, I ensure the manifest.json has correct metadata: unique UUID, version following semver, accurate SDK version requirements, and complete action definitions with icons. Then I compile TypeScript to JavaScript and ensure all dependencies are bundled. I run streamdeck pack com.myname.plugin.sdPlugin which creates a .streamDeckPlugin file. For marketplace submission, I also need marketing assets: a 600x400 hero image, 1-3 screenshots, a detailed description, and a support URL. I submit through the Elgato Developer Program portal, where it goes through review. For updates, I increment the version number and submit again.”
5. “What’s your strategy for testing a Stream Deck plugin?”
Strong answer: “I use a layered testing approach. Unit tests cover the macro parsing, condition evaluation, and step execution logic—these don’t need the Stream Deck at all. I mock the AutomationAdapter to test the execution engine in isolation. Integration tests run against the real SDK using a simulated action context. For end-to-end testing, I have a set of test macros that exercise different step types and conditionals. I use console.log and the Stream Deck debug logs during development. For cross-platform testing, I use VMs or GitHub Actions with macOS and Windows runners.”
6. “How do you handle errors when a step fails mid-execution?”
Strong answer: “Each step has a configurable error handling policy: ‘abort’, ‘skip’, or ‘retry’. The default is ‘abort’. When a step throws, the engine catches it, logs the error with context (step index, type, and error message), updates the key to show an error state, and either aborts the remaining steps or continues based on the policy. For ‘retry’, I implement exponential backoff up to 3 attempts. The macro JSON can specify per-step error handling. After any error, the engine emits an ‘error’ event that the action can use for user notification. The key shows which step failed.”
7. “How do you prevent your macro engine from being exploited for malicious automation?”
Strong answer: “Security is layered. First, macros are stored in the plugin’s sandboxed data directory, not user-accessible locations. Import validates JSON strictly against the schema and rejects unknown step types. For sensitive operations like ‘run shell command’, I require explicit user confirmation in the Property Inspector before enabling. I sanitize all user-provided text before passing to shell commands to prevent injection. Rate limiting prevents rapid-fire execution. I log all automation actions for auditability. For marketplace distribution, Elgato’s review process adds another layer of validation.”
8. “How would you add support for recording macros by capturing user actions?”
Strong answer: “Recording is complex because it requires system-level keyboard and mouse hooking. On macOS, this requires Accessibility permissions. I’d use the iohook npm package for cross-platform input capture. When recording starts, I install hooks for key events and mouse clicks. Each event is translated to a macro step: keystrokes become ‘keystroke’ steps, clicks become ‘click’ steps with coordinates. I detect application focus changes to insert ‘focus’ steps. The raw events are buffered and then processed to merge consecutive typing into ‘type’ steps and filter out navigation noise. Recording stops on a designated key combo or Stream Deck button press.”
Hints in Layers
Reveal these progressively as you work through the project:
Hint 1: Cross-Platform Automation Adapter
The core of cross-platform automation is the Adapter pattern:
// automation/AutomationAdapter.ts
export interface AutomationAdapter {
launchApp(appName: string): Promise<void>;
sendKeystroke(keys: string): Promise<void>;
typeText(text: string): Promise<void>;
getActiveApp(): Promise<{ name: string; title: string } | null>;
isAppRunning(appName: string): Promise<boolean>;
}
// automation/MacOSAdapter.ts
import { exec } from 'child_process';
import { promisify } from 'util';
const execAsync = promisify(exec);
export class MacOSAdapter implements AutomationAdapter {
async launchApp(appName: string): Promise<void> {
await execAsync(`open -a "${appName}"`);
}
async sendKeystroke(keys: string): Promise<void> {
// Convert "cmd+k" to AppleScript format
const { modifiers, key } = this.parseKeys(keys);
const modifierStr = modifiers.map(m => `${m} down`).join(', ');
const script = `
tell application "System Events"
keystroke "${key}" using {${modifierStr}}
end tell
`;
await execAsync(`osascript -e '${script}'`);
}
async typeText(text: string): Promise<void> {
// Escape special characters for AppleScript
const escaped = text.replace(/"/g, '\\"').replace(/'/g, "'\\''");
const script = `
tell application "System Events"
keystroke "${escaped}"
end tell
`;
await execAsync(`osascript -e '${script}'`);
}
async getActiveApp(): Promise<{ name: string; title: string } | null> {
const script = `
tell application "System Events"
set frontApp to name of first process whose frontmost is true
set frontWindow to ""
try
set frontWindow to name of front window of (first process whose frontmost is true)
end try
return frontApp & "|" & frontWindow
end tell
`;
const { stdout } = await execAsync(`osascript -e '${script}'`);
const [name, title] = stdout.trim().split('|');
return { name, title: title || '' };
}
async isAppRunning(appName: string): Promise<boolean> {
const script = `
tell application "System Events"
return (name of processes) contains "${appName}"
end tell
`;
const { stdout } = await execAsync(`osascript -e '${script}'`);
return stdout.trim() === 'true';
}
private parseKeys(keys: string): { modifiers: string[]; key: string } {
const parts = keys.toLowerCase().split('+');
const key = parts.pop() || '';
const modifiers = parts.map(m => {
switch (m) {
case 'cmd': return 'command';
case 'ctrl': return 'control';
case 'alt': return 'option';
case 'shift': return 'shift';
default: return m;
}
});
return { modifiers, key };
}
}
Hint 2: Macro JSON Schema Design
Design your schema with extensibility in mind:
// types/Macro.ts
interface MacroStep {
type: string;
id: string; // Unique ID for drag-drop and error reporting
errorPolicy?: 'abort' | 'skip' | 'retry';
retryCount?: number;
timeout?: number; // ms
}
interface LaunchStep extends MacroStep {
type: 'launch';
app: string;
waitForFocus?: boolean;
}
interface KeystrokeStep extends MacroStep {
type: 'keystroke';
keys: string; // e.g., "cmd+shift+a"
}
interface TypeStep extends MacroStep {
type: 'type';
text: string;
delayBetweenChars?: number; // ms
}
interface DelayStep extends MacroStep {
type: 'delay';
ms: number;
}
interface ConditionalStep extends MacroStep {
type: 'conditional';
condition: Condition;
thenSteps: MacroStep[];
elseSteps?: MacroStep[];
}
interface Condition {
type: 'appRunning' | 'appFocused' | 'timeAfter' | 'timeBefore' | 'clipboardContains' | 'and' | 'or';
// Type-specific fields
app?: string;
time?: string; // "18:00"
pattern?: string; // regex for clipboard
conditions?: Condition[]; // for and/or
}
interface Macro {
id: string;
name: string;
version: number; // Schema version for migrations
description?: string;
icon?: string; // Custom icon data URL
preconditions?: Condition[];
steps: MacroStep[];
settings: {
showProgress: boolean;
playSound: boolean;
successMessage?: string;
errorMessage?: string;
};
}
Hint 3: Macro Execution Engine
The engine orchestrates step execution:
// engine/MacroRunner.ts
import { EventEmitter } from 'events';
type RunnerState = 'idle' | 'evaluating' | 'executing' | 'complete' | 'error';
export class MacroRunner extends EventEmitter {
private state: RunnerState = 'idle';
private currentStepIndex = 0;
private adapter: AutomationAdapter;
private stepExecutor: StepExecutor;
private conditionEvaluator: ConditionEvaluator;
constructor(adapter: AutomationAdapter) {
super();
this.adapter = adapter;
this.stepExecutor = new StepExecutor(adapter);
this.conditionEvaluator = new ConditionEvaluator(adapter);
}
async execute(macro: Macro): Promise<MacroResult> {
this.state = 'evaluating';
this.emit('stateChange', { state: this.state });
// Check preconditions
if (macro.preconditions?.length) {
const conditionsMet = await this.evaluateConditions(macro.preconditions);
if (!conditionsMet) {
this.state = 'complete';
this.emit('conditionsNotMet', { macro });
return { success: false, reason: 'conditions_not_met' };
}
}
// Execute steps
this.state = 'executing';
this.emit('stateChange', { state: this.state });
for (let i = 0; i < macro.steps.length; i++) {
this.currentStepIndex = i;
const step = macro.steps[i];
this.emit('stepStart', {
index: i,
total: macro.steps.length,
step
});
try {
await this.executeStepWithPolicy(step);
this.emit('stepComplete', { index: i, step });
} catch (error) {
this.emit('stepError', { index: i, step, error });
if (step.errorPolicy === 'abort' || !step.errorPolicy) {
this.state = 'error';
return {
success: false,
reason: 'step_failed',
failedStep: i,
error
};
}
// 'skip' continues to next step
}
}
this.state = 'complete';
this.emit('stateChange', { state: this.state });
this.emit('complete', { macro });
return { success: true };
}
private async executeStepWithPolicy(step: MacroStep): Promise<void> {
const timeout = step.timeout || 10000;
const maxRetries = step.retryCount || 3;
let lastError: Error | null = null;
for (let attempt = 0; attempt < maxRetries; attempt++) {
try {
await Promise.race([
this.stepExecutor.execute(step),
this.createTimeout(timeout)
]);
return; // Success!
} catch (error) {
lastError = error as Error;
if (step.errorPolicy !== 'retry' || attempt === maxRetries - 1) {
throw error;
}
// Wait before retry (exponential backoff)
await this.delay(Math.pow(2, attempt) * 1000);
}
}
}
private createTimeout(ms: number): Promise<never> {
return new Promise((_, reject) => {
setTimeout(() => reject(new Error('Step timeout')), ms);
});
}
private delay(ms: number): Promise<void> {
return new Promise(resolve => setTimeout(resolve, ms));
}
private async evaluateConditions(conditions: Condition[]): Promise<boolean> {
for (const condition of conditions) {
const result = await this.conditionEvaluator.evaluate(condition);
if (!result) return false;
}
return true;
}
}
Hint 4: Active Application Detection
Use active-win for simple cases, or shell commands for more control:
// Using active-win (simpler)
import activeWin from 'active-win';
async function getActiveApp(): Promise<{ name: string; title: string } | null> {
const win = await activeWin();
if (!win) return null;
return {
name: win.owner.name,
title: win.title
};
}
// Using shell commands (more control)
async function isAppRunning(appName: string): Promise<boolean> {
if (process.platform === 'darwin') {
// macOS: use pgrep
try {
await execAsync(`pgrep -x "${appName}"`);
return true;
} catch {
return false;
}
} else if (process.platform === 'win32') {
// Windows: use tasklist
try {
const { stdout } = await execAsync(
`tasklist /FI "IMAGENAME eq ${appName}.exe" /NH`
);
return stdout.includes(appName);
} catch {
return false;
}
}
return false;
}
Hint 5: Property Inspector Drag-and-Drop Editor
Use Sortable.js for the visual macro editor:
<!-- ui/macro-editor.html -->
<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" href="css/sdpi.css">
<link rel="stylesheet" href="css/editor.css">
</head>
<body>
<div class="sdpi-wrapper">
<div class="sdpi-item">
<label class="sdpi-item-label">Macro Name</label>
<input type="text" id="macroName" class="sdpi-item-value">
</div>
<div class="sdpi-item">
<label class="sdpi-item-label">Steps</label>
<div id="stepsList" class="steps-container">
<!-- Steps rendered here by JavaScript -->
</div>
<button id="addStep" class="add-step-btn">+ Add Step</button>
</div>
</div>
<script src="js/sortable.min.js"></script>
<script src="js/sdpi.js"></script>
<script>
let macro = { name: '', steps: [] };
// Initialize Sortable for drag-and-drop
const stepsList = document.getElementById('stepsList');
Sortable.create(stepsList, {
animation: 150,
handle: '.drag-handle',
onEnd: () => {
// Reorder steps array based on DOM order
const stepElements = stepsList.querySelectorAll('.step-item');
const reordered = Array.from(stepElements).map(el => {
return macro.steps.find(s => s.id === el.dataset.stepId);
});
macro.steps = reordered;
saveToPlugin();
}
});
function renderSteps() {
stepsList.innerHTML = macro.steps.map((step, i) => `
<div class="step-item" data-step-id="${step.id}">
<span class="drag-handle">☰</span>
<span class="step-number">${i + 1}</span>
<select class="step-type" data-index="${i}">
<option value="launch" ${step.type === 'launch' ? 'selected' : ''}>Launch App</option>
<option value="keystroke" ${step.type === 'keystroke' ? 'selected' : ''}>Keystroke</option>
<option value="type" ${step.type === 'type' ? 'selected' : ''}>Type Text</option>
<option value="delay" ${step.type === 'delay' ? 'selected' : ''}>Delay</option>
<option value="conditional" ${step.type === 'conditional' ? 'selected' : ''}>Conditional</option>
</select>
${renderStepConfig(step, i)}
<button class="delete-step" data-index="${i}">✕</button>
</div>
`).join('');
}
function renderStepConfig(step, index) {
switch (step.type) {
case 'launch':
return `<input type="text" class="step-config" data-index="${index}"
data-field="app" value="${step.app || ''}"
placeholder="App name (e.g., Slack)">`;
case 'keystroke':
return `<input type="text" class="step-config" data-index="${index}"
data-field="keys" value="${step.keys || ''}"
placeholder="Keys (e.g., cmd+k)">`;
case 'type':
return `<input type="text" class="step-config" data-index="${index}"
data-field="text" value="${step.text || ''}"
placeholder="Text to type">`;
case 'delay':
return `<input type="number" class="step-config" data-index="${index}"
data-field="ms" value="${step.ms || 1000}"
placeholder="Milliseconds">`;
default:
return '';
}
}
function saveToPlugin() {
$SD.api.setSettings({ macro });
}
// Event listeners for add, delete, type change, config change...
document.getElementById('addStep').addEventListener('click', () => {
macro.steps.push({
id: crypto.randomUUID(),
type: 'launch',
app: ''
});
renderSteps();
saveToPlugin();
});
$SD.on('didReceiveSettings', (ev) => {
macro = ev.payload.settings.macro || { name: '', steps: [] };
document.getElementById('macroName').value = macro.name;
renderSteps();
});
</script>
</body>
</html>
Hint 6: Plugin Packaging and Distribution
Package your plugin for distribution:
# 1. Build your TypeScript
npm run build
# 2. Ensure manifest.json is correct
cat com.yourname.macrosuite.sdPlugin/manifest.json
# Check: UUID, Version, CodePath, Actions, Icons
# 3. Pack the plugin
streamdeck pack com.yourname.macrosuite.sdPlugin
# Output: com.yourname.macrosuite.streamDeckPlugin
# 4. Test by double-clicking the .streamDeckPlugin file
# It should install into Stream Deck automatically
# 5. For Marketplace submission:
# - Create 600x400 hero image
# - Take 1-3 screenshots (1920x1080 recommended)
# - Write description (supports Markdown)
# - Set up support URL (GitHub issues page works)
# - Submit via: https://developer.elgato.com/
Your manifest.json should include:
{
"Name": "Macro Automation Suite",
"Version": "1.0.0",
"Author": "Your Name",
"Description": "Build powerful multi-step automations with conditional logic",
"Category": "Utility",
"UUID": "com.yourname.macrosuite",
"Icon": "assets/plugin-icon",
"CodePath": "bin/plugin.js",
"Software": {
"MinimumVersion": "6.4"
},
"Nodejs": {
"Version": "20",
"Debug": "enabled"
},
"Actions": [
{
"Name": "Run Macro",
"UUID": "com.yourname.macrosuite.execute",
"Icon": "assets/action-run",
"Tooltip": "Execute a saved macro sequence",
"PropertyInspectorPath": "ui/macro-editor.html",
"SupportedControllers": ["Keypad"],
"States": [
{ "Image": "assets/state-idle" },
{ "Image": "assets/state-running" }
]
}
]
}
Books That Will Help
This capstone draws on many disciplines. These books will give you the deep understanding needed:
| Topic | Book | Chapter | Why It Helps |
|---|---|---|---|
| Design Patterns | Node.js Design Patterns by Casciaro & Mammino | Ch. 8: Structural Patterns (Adapter, Proxy) | The Adapter pattern is how you abstract platform differences. Essential reading. |
| Async Patterns | Node.js Design Patterns by Casciaro & Mammino | Ch. 5: Asynchronous Control Flow Patterns | Your macro engine runs async steps sequentially with error handling. This chapter is a goldmine. |
| Process Management | Node.js Design Patterns by Casciaro & Mammino | Ch. 9: Behavioral Patterns | Child process spawning, event emitters, and the patterns that make your engine robust. |
| Workflow Engines | Enterprise Integration Patterns by Hohpe & Woolf | Process Manager pattern (Ch. 7) | Your macro runner is a simple workflow engine. This pattern defines the architecture. |
| State Machines | Designing Data-Intensive Applications by Kleppmann | Ch. 11: Stream Processing | Understanding event-driven state machines at scale informs your execution engine design. |
| Schema Design | Designing Data-Intensive Applications by Kleppmann | Ch. 4: Encoding and Evolution | Your macro JSON format needs versioning and backward compatibility. This chapter explains how. |
| Testing Strategies | The Pragmatic Programmer by Thomas & Hunt | Topic 41: Test to Code | Layered testing (unit → integration → e2e) for plugin development. |
| Decoupling | The Pragmatic Programmer by Thomas & Hunt | Topic 28: Decoupling | Keeping your automation adapter, execution engine, and UI cleanly separated. |
| Error Handling | Release It! by Nygard | Ch. 4-5: Stability Patterns | Timeouts, circuit breakers, and bulkheads apply directly to step execution. |
| Child Processes | The Linux Programming Interface by Kerrisk | Ch. 24-28: Process Creation | Deep understanding of fork/exec, signals, and process lifecycle—useful for AppleScript/PowerShell spawning. |
| AppleScript | AppleScript: The Definitive Guide by Neuburg | Ch. 10-15: System Events | If you’re on macOS, this is how you’ll automate other apps. |
| PowerShell | Learn PowerShell in a Month of Lunches by Jones & Hicks | Ch. 7-10: Commands and Scripting | Windows automation backbone—essential for cross-platform. |
| UI Components | JavaScript: The Good Parts by Crockford | Ch. 4-5: Functions & Objects | Property Inspector is vanilla JS. Clean patterns matter in constrained environments. |
| Plugin Architecture | Clean Architecture by Martin | Part V: Architecture | Structuring your plugin for testability and future feature additions. |
Prioritized Reading Order:
- First: Node.js Design Patterns Ch. 5, 8, 9 → Your foundation for async and patterns
- Second: Enterprise Integration Patterns Process Manager → Your execution engine architecture
- Third: Designing Data-Intensive Applications Ch. 4 → Your macro JSON schema
- Fourth: The Pragmatic Programmer Topics 28, 41 → Testing and clean separation
- As needed: Platform-specific books (AppleScript or PowerShell) based on your target
Getting Started Right Now
# 1. Install Node.js 20+ (if needed)
brew install node # macOS
# 2. Install Stream Deck CLI
npm install -g @elgato/cli
# 3. Create your first plugin
streamdeck create
# 4. Follow the wizard, then:
cd your-plugin-name
npm install
npm run watch
Open Stream Deck → Your new action appears → Start building!