Project 7: Signal Mesh for Multi-Agent Delegation

Quick Reference

Attribute	Value
Difficulty	4
Time	2-3 weeks
Main Stack	jido + signal-driven routing
Alternatives	direct function calls or shared queue only
Why Now	Multi-agent systems fail without envelope discipline

What You Will Build

A signal mesh where a coordinator agent delegates tasks to worker agents via typed signals, with backpressure and parent-child routing.

Real World Outcome

$ mix run -e "SignalMeshDemo.simulate(:triage)"
[info] mesh_id=mesh-11
[info] coordinator=triage_master routed=12 tasks pending=0
[info] workers=[analysis_a, analysis_b, policy_c]
[info] delegations=12 complete=12 failed=0 deadline_ms=3000

The resulting trace should show:

coordinator emits delegation signals
worker agents process in parallel
structured completion signals are collected and summarized

The Core Question You Are Answering

“How do you scale collaborative agents while preserving determinism and bounded communication?”

Why This Project Matters

Jido’s signal abstraction formalizes message envelopes across agents. This is the practical difference between brittle callback sprawl and coherent multi-agent delegation.

Mesh Diagram

                 +---------------------+
                 |    Coordinator       |
                 |  (planner agent)     |
                 +----+----------+------+
                      |          |
                      | signals  | signals
                      v          v
         +----------------+  +----------------+
         | Worker: Parse  |  | Worker: Policy |
         | - create tasks |  | - classify     |
         +-------+--------+  +-------+--------+
                 |                   |
                 +---------+ +-------+
                           | |
                           v v
                        +--------+
                        |  Bus   |
                        | (routing layer) |
                        +---+----+----+
                            |    |
                            |    +------------------+
                            v                      v
                   +---------------+       +---------------+
                   | Worker: Output|       | Worker: Guard  |
                   | Summarizer    |       | Validator     |
                   +-------+-------+       +-------+-------+
                           \_______________________/
                                           |
                                           v
                                  +------------------+
                                  | Completion Signals|
                                  | Coordinator Merge |
                                  +------------------+

Deep Dive Plan

1. Message Envelope

Use a canonical signal shape:

type
data
source
correlation_id
schema_version

2. Delegation Strategy

Coordinator decides worker assignment by:

skill domain
expected latency
current queue pressure

3. Routing Rules

Route map includes:

priority rules
fanout vs direct route
timeout and deadline policies

4. Merge/Reduce

Coordinator receives completion signals and resolves:

result precedence
confidence ordering
conflict arbitration

5. Backpressure & Retry

per-worker in-flight caps
deadline propagation
failed worker routes to fallback worker

6. Optional LLM step

Use req_llm as classification aid in coordinator action:

normalize noisy tasks before routing
detect impossible tasks early

Concepts You Must Understand First

Signal contracts
- Why consistent envelopes prevent lost context.
Fanout governance
- Not every task needs full broadcast.
Parent-child tracking
- Keep lineage for audit and cost attribution.
Backpressure
- control queue health before worker overload.

Questions to Guide Your Design

Delegation
- Which tasks are parallelizable vs sequential?
Aggregation
- How do you merge contradictory worker outputs?
Failure
- What counts as partial success?

Thinking Exercise

Draw one message timeline for: triage -> classify -> summarize -> publish.

Include:

worker IDs
completion order
which signal carries override rights

Now inject a late worker signal that conflicts with earlier summary. What should arbitration do?

Interview Questions They Will Ask

“How does signal typing help multi-agent collaboration?”
“How do you implement backpressure in a mesh?”
“What is your strategy for contradictory worker outputs?”
“How does coordinator avoid becoming a single point of failure?”
“How do correlation IDs improve auditability?”

Hints in Layers

Hint 1: Define a strict message schema Use explicit type and correlation_id early.

Hint 2: Add route registry Map task types to worker sets.

Hint 3: Start with synchronous fanout Then introduce async + buffering once protocol is stable.

Hint 4: Instrument every handoff Each handoff should have one trace event.

Common Pitfalls and Debugging

Problem: Orphaned signals and no completion.
- Why: worker not acknowledging timeout boundary.
- Fix: TTL + explicit completion or failure signals.
- Quick test: run with one disabled worker.
Problem: Flooded coordinator.
- Why: no in-flight limit.
- Fix: apply worker concurrency quotas.
- Quick test: stress test with 10x expected load.
Problem: Inconsistent arbitration.
- Why: non-deterministic merge order.
- Fix: stable ordering by timestamp+source+version.
- Quick test: replay same signal set and compare merged output hash.

Books That Will Help

Topic	Book	Chapter
Coordination patterns	Building Event-Driven Systems	Event contracts and reliability
Fault tolerance	Seven Concurrency Models in Erlang	Message passing and process load

Definition of Done

Signal mesh routes tasks according to typed envelopes
Worker concurrency is bounded and observable
completion arbitration is deterministic
failure and fallback routes are explicit
trace supports replay and debugging

References

https://hexdocs.pm/jido/2.0.0-rc.4/readme.html
https://hexdocs.pm/jido/2.0.0-rc.4/readme.html#Core%20Concepts