Project 2: Registry Topic Bus
Replace the hand-maintained subscriber list with a supervised, local topic bus whose memberships are process-owned duplicate Registry entries.
Quick Reference
| Attribute | Value |
|---|---|
| Difficulty | Level 2: Intermediate |
| Time Estimate | 8–12 hours |
| Language | Elixir (alternatives: Erlang, Gleam) |
| Prerequisites | Project 1, OTP application basics, supervisors, process lifecycle, ExUnit |
| Key Topics | Duplicate Registry keys, process-owned membership, dispatch partitions, idempotent subscribe, metadata, local routing |
| Primary Tools | Elixir Registry, Supervisor, IEx, ExUnit |
| Produces | A supervised node-local topic bus with diagnostics and bounded dispatch |
1. Learning Objectives
By completing this project, you will:
- Explain why a duplicate Registry is a useful local membership index for Pub/Sub.
- Contrast a Registry-backed bus with a central GenServer holding a topic-to-PID map.
- Define explicit semantics for repeated subscription rather than accepting accidental duplicate delivery.
- Explain how registry entries belong to the registering process and disappear when that process exits.
- Use Registry dispatch without assuming one global callback invocation or one ordered subscriber list.
- Keep dispatch callbacks bounded so publisher latency is not dominated by filtering, I/O, or business logic.
- Store compact routing metadata while keeping authoritative business state elsewhere.
- Design introspection and metrics that do not require expensive full-registry scans or unbounded topic labels.
- State that Elixir Registry is node-local and does not create a distributed Pub/Sub system.
2. Theoretical Foundation
2.1 Core Concepts
Registry as a process-owned index
Elixir Registry associates keys and values with the process that performs the registration. A unique Registry acts like a name table: one key points to at most one process. A duplicate Registry acts like a property or membership table: many entries can share the same key. Pub/Sub maps naturally onto duplicate keys because a topic is the shared key and each subscriber process contributes one membership entry.
The ownership rule matters. A separate broker process does not own the membership rows on behalf of subscribers. If a subscriber exits, Registry removes its entries. This eliminates the stale-membership table that a hand-built broker would otherwise need to maintain with monitors. It does not guarantee that a PID returned by a lookup will still be alive one instruction later, and it does not preserve messages or subscriptions across a process restart. Membership cleanup and message delivery are different properties.
Duplicate registration semantics
Duplicate Registry mode permits multiple processes under one key and also permits repeated registrations by the same process under that key. That behavior is a primitive, not the correct domain API for every bus. A consumer that calls subscribe twice may expect idempotence, while another design may treat each registration as a distinct filter or handler. The wrapper must choose and test one meaning. This project chooses idempotent topic subscription: one process receives at most one copy of a publication for a topic, regardless of repeated subscribe calls.
Partitions and dispatch
A Registry can be partitioned to reduce contention. With the traditional duplicate-key configuration, entries for one topic can be spread across partitions according to subscriber PID. Dispatch invokes the supplied callback with the entries found in each relevant partition; it must not assume one complete list. Depending on the options and Registry version, partition work may happen serially or in parallel. Even serial partition traversal does not create a domain ordering guarantee among subscribers because sends enter independent mailboxes.
Dispatch executes as part of the publishing operation. A callback that performs expensive filtering, logging, network I/O, database access, or subscriber business work turns publication into a slow caller-path operation. The correct callback is small: validate already-compact metadata, send the event, and collect bounded counters. Subscriber-specific work runs in subscriber processes after the send.
Registry is local
Registry indexes processes on one BEAM node. The same Registry name on another node represents different tables and different memberships. Building a distributed bus requires another layer that transports broadcasts between nodes and invokes local fanout at each destination. Phoenix.PubSub supplies that boundary later. This project must label its scope as node-local.
2.2 Central Broker Versus Decentralized Membership
A central broker often starts with one state map:
state = topic -> set of subscriber PIDs
Subscribe, unsubscribe, publish, and cleanup requests all pass through one process. This design is easy to understand and can support sophisticated ordering or policy, but the process is a serialization point. It also owns volatile membership state that must be rebuilt after a crash. If it fails to monitor subscribers correctly, stale PIDs remain.
Registry distributes the index across ETS-backed partitions and ties rows to subscriber processes:
subscriber process Registry partition
| |
| register(topic, metadata) |
|----------------------------->|
| membership owned by PID |
| |
X process exits |
| remove PID entries
Publication is a lookup-and-dispatch operation rather than a request to a broker loop:
publisher
|
| dispatch topic
v
+---------------- Registry ----------------+
| partition 1 partition 2 partition N |
| [P1, meta] [P2, meta] [P3, meta] |
+-------+-------------+-------------+-------+
| | |
v v v
mailbox mailbox mailbox
The trade is not “Registry is always better.” A central owner is appropriate when publication itself must serialize transitions, apply complex admission rules, or persist membership. Registry is appropriate for volatile local routing where process ownership, concurrent lookup, and automatic cleanup are valuable.
2.3 Membership Is Not Delivery
The following events can all occur:
- Dispatch finds a registered PID.
- The process exits immediately afterward.
- The publisher sends to that old PID.
- Registry removes the stale entry.
- Publish returns successfully from its routing operation.
- No handler receipt exists.
Registry narrows the stale-membership window but cannot turn asynchronous messaging into processing acknowledgement. The bus returns a routing summary such as entries observed and sends issued. It never returns “all subscribers processed.”
Likewise, registration is tied to one PID incarnation. If a supervised subscriber restarts, its replacement must subscribe again during initialization. A subscription is not a durable consumer identity and the Registry is not a replay log.
2.4 Subscription Metadata
Registry values can carry compact metadata used during dispatch:
SubscriptionMetadata:
logical_subscriber: ui_projection
accepted_version_range: 1..2
filter_tag: west_region
instrumentation_class: interactive
Useful metadata is immutable or cheaply replaceable, small enough to copy and inspect, and independent of secrets. Avoid storing full user records, authorization snapshots, large filter functions, or mutable business state. Authorization should happen before a topic is derived or before a subscriber is admitted. Registry metadata may help route already-authorized messages; it must not become an unreviewed policy database.
2.5 Mental Model
Application Supervisor
|
+-- LocalTopics Registry
| keys: duplicate
| partitions: configured
|
+-- Subscriber Supervisor
|
+-- West UI process ----- registers sensor:west
+-- West alert process -- registers sensor:west
+-- East UI process ----- registers sensor:east
publish(sensor:west, event)
|
v
Registry.dispatch
/ \
v v
West UI PID West alert PID
mailbox mailbox
East UI receives nothing.
West alert exits
|
v
Registry removes its process-owned entries
|
v
next west dispatch observes one membership
2.6 Definitions and Key Terms
- Registry: A local process index associating process-owned keys with values.
- Unique keys: Registry mode where one key maps to at most one process.
- Duplicate keys: Registry mode where a key can have multiple process-owned entries.
- Registration owner: The process that called register; its lifecycle controls the entry.
- Partition: One shard of Registry’s internal storage and dispatch work.
- Dispatch callback: A function invoked with entries for the requested key in one partition.
- Idempotent subscription: Repeating subscribe has the same externally visible membership effect as calling it once.
- Routing metadata: Small values attached to a membership and used to choose or describe delivery.
- Node-local: Available only inside one Erlang node.
- Stale-PID race: The unavoidable interval in which a looked-up process can exit before a send.
2.7 Minimal Concrete Example
Protocol outline:
SUBSCRIBE(topic, metadata):
validate topic and metadata
if current process already has logical membership:
return ALREADY_SUBSCRIBED
otherwise:
register current process under duplicate key topic
return NEW
PUBLISH(topic, event):
validate event
dispatch topic
for each partition callback:
for each [pid, metadata]:
if compact filter accepts:
send EVENT to pid
increment sends_issued
return ROUTING_SUMMARY
UNSUBSCRIBE(topic):
remove current process registrations for topic
return REMOVED or NOT_SUBSCRIBED according to wrapper policy
The outline intentionally contains no business handler and no processing acknowledgement.
2.8 Common Misconceptions
- “Duplicate Registry prevents the same PID from registering twice.” Raw duplicate mode allows duplicates; the wrapper must impose idempotence if desired.
- “Automatic cleanup means lookup and send are atomic with process life.” A process can exit after lookup.
- “Dispatch gives one complete subscriber list.” Partitioned dispatch may invoke the callback multiple times.
- “Dispatch callback work runs in each subscriber.” Routing callback work is part of the publisher path or spawned partition tasks; business work belongs after send.
- “Registry is distributed.” It is local to one node.
- “A Registry value is a good place for complete session state.” It should remain compact routing metadata.
- “Parallel dispatch establishes a global order.” It does the opposite: it increases concurrency and still cannot order independent mailboxes.
2.9 Check Your Understanding
- Why does duplicate Registry fit topic membership?
- Who owns a registration and what removes it?
- Why must the wrapper define repeated-subscribe behavior?
- Why can dispatch invoke the callback more than once?
- Which work is unsafe inside the callback?
- What guarantee does a routing summary not provide?
- What must a restarted subscriber do?
Answers
- One topic key can point to many subscriber PIDs.
- The registering process owns it; Registry removes it when that process exits.
- Raw duplicate registration can produce repeated entries and duplicate sends.
- Entries may live in multiple partitions, each dispatched independently.
- Blocking I/O, expensive computation, unbounded logging, and business handlers.
- It does not prove that any subscriber handled the event.
- Register the new PID again; the old subscription and mailbox do not transfer.
2.10 Why This Matters
Many Elixir systems need a lightweight, local observer mechanism without adding a broker process to every topic. Registry offers scalable local indexing and lifecycle cleanup, but only if the application respects its scope. Learning the primitive makes Phoenix.PubSub less magical: you will recognize local subscription storage, fanout, partition sizing, and the remaining need for an adapter when a broadcast crosses nodes.
2.11 References
- Elixir Registry — duplicate keys, registration ownership, partitions, and dispatch.
- Elixir Supervisor — supervising Registry as application infrastructure.
- Elixir Process — process lifecycle and inspection.
- “Enterprise Integration Patterns” by Gregor Hohpe and Bobby Woolf — Publish-Subscribe Channel.
- “Designing for Scalability with Erlang/OTP” — process and supervision architecture.
3. Project Specification
3.1 What You Will Build
Build a supervised local topic bus named LocalTopics. Subscriber processes register themselves under duplicate topic keys. A thin domain-neutral wrapper validates topic forms, makes repeated subscription idempotent, stores compact metadata, and exposes subscribe, unsubscribe, publish, membership count, and diagnostic snapshot operations.
The project includes three IEx subscriber fixtures, a churn scenario, a quiet benchmark profile, and tests for duplicates, cleanup, partition behavior, and publisher-path boundedness. It does not include cross-node propagation, durable replay, or authorization policy.
3.2 Functional Requirements
- Supervised infrastructure: Registry starts under the application supervisor before subscriber children.
- Duplicate topic keys: Several processes can subscribe to the same topic.
- Idempotent wrapper: One process receives at most one copy after repeated logical subscription.
- Process-owned unsubscribe: A subscriber removes its own membership.
- Automatic exit cleanup: Dead subscriber entries disappear without a separate cleanup table.
- Compact metadata: Subscriber class and a bounded filter tag may be stored.
- Partition-safe dispatch: The callback handles partial entry batches correctly.
- Bounded callback: Dispatch performs validation, a simple filter, send, and bounded counting only.
- Diagnostic summary: Report topic, entries observed, sends issued, partitions touched, and duration.
- No acknowledgement claim: The return type labels routing completion only.
- Node-local label: Logs and documentation state that delivery is local.
- Churn test: Repeated process start/exit does not leave ghost memberships.
3.3 Non-Functional Requirements
- Performance: Publish cost should scale approximately with matching entries, not total registry size.
- Reliability: A malformed metadata entry is rejected at subscription or skipped with a metric; it does not crash all fanout.
- Observability: Metrics use bounded subscriber classes and topic families, not raw tenant IDs.
- Usability: Subscription results distinguish NEW from ALREADY_SUBSCRIBED.
- Maintainability: Application code calls the wrapper, not Registry directly.
- Scope clarity: No claim of cross-node, durable, ordered, or acknowledged delivery.
3.4 Example Usage and Exact Output
$ mix run --no-halt
bus=LocalTopics scope=node_local registry_partitions=4 status=ready
subscribe pid=<0.181.0> topic=sensor:west result=new
subscribe pid=<0.182.0> topic=sensor:west result=new
subscribe pid=<0.183.0> topic=sensor:east result=new
subscribe pid=<0.181.0> topic=sensor:west result=already_subscribed
publish topic=sensor:west entries_observed=2 sends_issued=2 partitions_touched=2 duration_us=41
handled pid=<0.181.0> event=evt-7 topic=sensor:west
handled pid=<0.182.0> event=evt-7 topic=sensor:west
exit pid=<0.182.0> reason=shutdown
membership topic=sensor:west count=1 cleanup=automatic
publish topic=sensor:west entries_observed=1 sends_issued=1 acknowledgement=none
3.5 Real World Outcome
Open three IEx sessions attached to one local node. Two processes join sensor:west and one joins sensor:east. Publishing a west event produces exactly two handler lines. Calling the wrapper’s subscribe operation again from the first west process returns already_subscribed, and the next publication still reaches it once.
Terminate the second west process. Without calling an application cleanup service, its Registry entry disappears. The next diagnostic snapshot shows one west member. Switching the benchmark profile from one partition to four shows how dispatch work is divided, but the report never uses callback order as a correctness guarantee. The final output clearly states scope=node_local and acknowledgement=none.
4. Solution Architecture
4.1 High-Level Design
+---------------- Application Supervisor ----------------+
| |
| +---------------- LocalTopics Registry -------------+ |
| | duplicate keys; N partitions; process-owned rows | |
| +--------------------+------------------------------+ |
| ^ |
| | register/unregister |
| +--------------------+------------------------------+ |
| | subscriber supervisor | |
| | west_ui west_alert east_ui | |
| +---------------------------------------------------+ |
+--------------------------------------------------------+
Publisher
|
| LocalTopicBus.publish(topic, event)
v
validate -> Registry.dispatch -> bounded filter -> async send
|
+------------+------------+
v v
subscriber PID subscriber PID
4.2 Key Components
| Component | Responsibility | Key Decisions |
|---|---|---|
| LocalTopics Registry | Own partitioned process membership index | Duplicate keys; explicitly configured partitions |
| TopicBus wrapper | Validate topics/events and define semantics | Only entry point used by application |
| Subscription helper | Make repeated logical subscribe idempotent | Raw Registry duplicates remain hidden |
| Dispatch function | Filter compact metadata and send | No I/O or business work |
| Diagnostics | Produce bounded counts and duration | Raw topic shown only in local CLI, not metric labels |
| Subscriber fixtures | Demonstrate independent handling and lifecycle | Re-register on process initialization |
| Benchmark scenario | Compare churn and fanout profiles | Trend evidence, not universal throughput claims |
4.3 Data Structures
Topic:
namespace: sensor
scope: west
canonical_text: sensor:west
SubscriptionMetadata:
subscriber_class: ui | alert | projection
filter_tag: bounded atom or short identifier
protocol_version: integer
Event:
id
type
schema_version
emitted_at_monotonic
payload_digest
RoutingSummary:
topic_family
entries_observed
sends_issued
rejected_entries
partitions_touched
duration_microseconds
acknowledgement: none
4.4 Algorithm Overview
Idempotent subscribe
- Validate the canonical topic.
- Validate metadata size and allowed fields.
- Determine whether this process already has the logical topic membership.
- If present, return ALREADY_SUBSCRIBED without another raw registration.
- Otherwise register under the duplicate key and return NEW.
Publish
- Validate topic and event outside Registry dispatch.
- Start a monotonic duration measurement.
- Dispatch entries for the key.
- For each callback batch, increment partitions touched.
- For each entry, apply only the compact metadata predicate.
- Send to accepted PIDs and count sends issued.
- Return the routing summary.
Complexity
- Subscribe/unsubscribe: approximately constant-time with partition lookup, subject to contention.
- Publish: O(M), where M is the number of entries for the topic examined.
- Memory: O(R), where R is total registrations.
- Full diagnostic scan: potentially O(R); keep it manual or rate-limited.
4.5 Architectural Invariants
| Invariant | Enforcement |
|---|---|
| Application cannot accidentally create duplicate logical subscriptions | Wrapper owns subscribe |
| Membership is volatile and process-owned | Registry rows only |
| Publish callback cannot perform I/O | Narrow internal callback contract and tests |
| Callback batches are independent | Counters merge across invocations |
| Cross-node delivery is not implied | Scope field and documentation |
| Routing completion is not processing completion | Explicit acknowledgement=none |
5. Implementation Guide
5.1 Development Environment Setup
$ elixir --version
Erlang/OTP 27 [...]
Elixir 1.18.[...]
$ mix new registry_topic_bus --sup
* creating mix.exs
* creating lib/...
* creating test/...
No external dependency is required for the core project. Use the Registry included with Elixir.
5.2 Suggested Project Structure
registry_topic_bus/
├── lib/
│ ├── registry_topic_bus/application.ex
│ └── registry_topic_bus/
│ ├── local_topics.ex
│ ├── topic.ex
│ ├── subscription_metadata.ex
│ ├── diagnostics.ex
│ └── subscriber_fixture.ex
├── lab/
│ ├── multi_session.exs
│ └── churn_profile.exs
├── test/
│ ├── subscription_test.exs
│ ├── dispatch_test.exs
│ ├── lifecycle_test.exs
│ └── partition_test.exs
└── README.md
5.3 The Core Question You Are Answering
“How can process-owned registrations replace a centralized subscription broker without pretending local membership is delivery?”
Answer this by drawing ownership. If a separate process owns the membership map, that process must observe subscriber death. If each subscriber owns its Registry entry, the Registry can remove it with process lifecycle. Neither design acknowledges handler completion merely by routing.
5.4 Concepts You Must Understand First
- Duplicate keys
- Can several PIDs share one key?
- Can one PID create repeated raw entries?
- Reference: Elixir Registry module documentation.
- Process-owned lifecycle
- Why does an entry disappear on process exit?
- Why can a just-looked-up PID still die?
- Book: “Designing for Scalability with Erlang/OTP,” process lifecycle.
- Dispatch execution
- Which caller pays for filtering and instrumentation?
- Why must callback work be bounded?
- Reference: Registry.dispatch documentation.
- Partitions
- Why can one logical dispatch have several callback batches?
- Why does batch order not define subscriber order?
- Reference: Registry options and dispatch.
- Supervision
- In which order should Registry and subscribers start?
- How does a restarted subscriber recover membership?
- Reference: Elixir Supervisor documentation.
5.5 Questions to Guide Your Design
- Is repeated subscribe idempotent or intentionally multiplicative?
- How does the wrapper recognize current-process membership without scanning the whole Registry?
- What topic forms are valid?
- Which metadata fields are bounded and safe?
- What happens if one metadata value is malformed?
- How will callbacks merge counts across partitions?
- What metrics can use topic family rather than raw topic?
- What should a restarted subscriber do during initialization?
- Which feature would justify returning to a central GenServer?
5.6 Thinking Exercise
Draw two implementations:
central GenServer: topic -> PID set
duplicate Registry: process-owned topic rows
For each, answer:
- Where is membership state?
- Which component is a serialization point?
- Who observes subscriber death?
- What happens after the membership component crashes?
- How is one topic published?
- Can one slow handler block routing?
- What is still volatile?
Then decide whether idempotence belongs in Registry itself or in your wrapper contract.
5.7 The Interview Questions They Will Ask
- How does Registry differ from a GenServer-owned map?
- Why are duplicate keys useful for Pub/Sub?
- What happens when a registered process exits?
- Where does Registry dispatch callback work execute?
- Is Registry distributed across nodes?
- How do partitions affect dispatch assumptions?
5.8 Hints in Layers
Hint 1: Supervise the Registry first
Put it before subscribers in the application supervision tree so initialization can register safely.
Hint 2: Hide raw registration
Make the wrapper the only application API and encode idempotence there.
Hint 3: Keep metadata boring
Store small values needed for bounded routing. Resolve everything else in the subscriber.
Hint 4: Benchmark churn, not just lookup
Exercise repeated subscriber start, subscribe, publish, and exit. Static lookup alone hides lifecycle costs.
5.9 Books That Will Help
| Topic | Book | Chapter or Section |
|---|---|---|
| Publish-Subscribe Channel | “Enterprise Integration Patterns” | Ch. 3 Messaging Channels |
| Process architecture | “Designing for Scalability with Erlang/OTP” | Processes and supervision |
| Stability | “Release It!, 2nd Edition” | Stability Patterns |
| Data ownership | “Programming Elixir” | OTP and process state chapters |
5.10 Implementation Phases
Phase 1: Supervised Registry — 2 hours
- Add duplicate Registry to the application supervision tree.
- Create two subscriber fixtures sharing a topic.
- Prove exit cleanup.
Checkpoint: A dead fixture disappears from lookup without application cleanup code.
Phase 2: Semantic Wrapper — 3 hours
- Define canonical topic and metadata validation.
- Make repeated logical subscribe idempotent.
- Define unsubscribe results.
Checkpoint: Subscribe twice, publish once, and observe exactly one copy.
Phase 3: Partition-Safe Dispatch — 3 hours
- Add bounded dispatch and merged routing counters.
- Exercise one and several partitions.
- Prohibit callback I/O.
Checkpoint: All matching subscribers receive one event regardless of callback batch count.
Phase 4: Diagnostics and Churn — 2 hours
- Add routing summaries and rate-limited snapshots.
- Run subscriber start/exit churn.
- Add topic-family metrics.
Checkpoint: No ghost membership remains after the churn scenario.
Phase 5: Testing and Documentation — 2–4 hours
- Add lifecycle, duplicate, malformed metadata, and no-subscriber tests.
- Document local-only scope and non-guarantees.
- Record representative profile output.
Checkpoint: The README never calls sends “processed deliveries.”
5.11 Key Implementation Decisions
| Decision | Options | Recommendation | Rationale |
|---|---|---|---|
| Membership owner | Broker process, subscriber process | Subscriber via Registry | Automatic lifecycle cleanup |
| Repeated subscribe | Duplicate copy, idempotent | Idempotent | Safer default for a topic bus |
| Registry mode | Unique, duplicate | Duplicate | One topic needs many PIDs |
| Metadata | Full session, compact routing fields | Compact fields | Keeps dispatch bounded |
| Callback work | Business handler, send only | Send plus bounded counters | Protects publisher latency |
| Metrics | Raw topics, topic families | Topic families | Avoids cardinality explosion |
| Distribution | Hide limitation, explicit local scope | Explicit local scope | Prevents false guarantees |
6. Testing Strategy
6.1 Test Categories
| Category | Purpose | Examples |
|---|---|---|
| Unit | Validate topic and metadata rules | Reject oversized or malformed values |
| Subscription | Validate wrapper semantics | NEW then ALREADY_SUBSCRIBED |
| Dispatch | Validate fanout and filters | Two west recipients, zero east recipients |
| Lifecycle | Validate automatic cleanup | Exit removes process-owned rows |
| Partition | Validate multiple callback batches | Merge counts without order assumptions |
| Churn | Detect ghost memberships | Start and stop many short-lived subscribers |
| Performance observation | Find caller-path regressions | Slow callback fixture is rejected |
6.2 Critical Test Cases
- Two PIDs share one duplicate key and each receives one event.
- One PID calls logical subscribe twice and receives one copy.
- One PID can subscribe to two distinct topics.
- Unsubscribe removes only that process’s membership for the topic.
- Process exit removes all of its Registry entries.
- Publish to an empty topic returns zero entries and zero sends.
- Malformed metadata is rejected before registration.
- Partitioned dispatch reaches all entries without relying on callback order.
- A subscriber exits between dispatch lookup and handling; no processing claim is made.
- Churn leaves the final membership count at zero.
6.3 Test Data
topics:
sensor:west
sensor:east
alerts:operations
subscribers:
west_ui: class=ui, filter=west
west_alert: class=alert, filter=west
east_ui: class=ui, filter=east
event:
id=evt-7
type=sensor.reading_changed
schema_version=1
expected:
publish sensor:west -> west_ui once, west_alert once
publish sensor:east -> east_ui once
duplicate logical subscribe by west_ui -> still one west copy
exit west_alert -> west membership count becomes one
6.4 Benchmark Discipline
- Warm the node before recording a profile.
- Compare like-for-like event and membership counts.
- Run one-partition and multi-partition profiles separately.
- Include subscription churn, not just publication.
- Disable verbose per-message logging in profile mode.
- Treat results as local evidence, not universal Registry limits.
7. Common Pitfalls & Debugging
7.1 Frequent Mistakes
| Pitfall | Symptom | Solution |
|---|---|---|
| Raw duplicate register exposed | One subscriber gets two copies | Enforce wrapper idempotence |
| Business work inside dispatch | Publish latency spikes | Send only; handle in subscriber |
| One callback expected | Missing counters or entries with partitions | Merge independent callback batches |
| Registry treated as distributed | Remote subscribers receive nothing | Label node-local; add adapter later |
| Full user state stored as metadata | Memory and dispatch cost grow | Store compact routing values |
| Subscriber restart not registering | Replacement receives nothing | Subscribe in new process initialization |
| Raw topic in every metric label | Metrics backend cardinality explodes | Use bounded topic family |
| Lookup used as liveness proof | Race causes missing handling | Accept lifecycle race; use protocol receipt if needed |
7.2 Debugging Strategies
- Compare Registry keys for a PID with expected logical memberships.
- Inspect lookup output for repeated PID-key entries when duplicate delivery occurs.
- Temporarily log callback batch sizes and partition count, not full payloads.
- Measure dispatch duration around the wrapper to expose expensive callbacks.
- Monitor subscriber fixture exits to distinguish cleanup from failed registration.
- Reduce to one partition to diagnose logic, then restore the production-like configuration.
7.3 Performance Traps
Full scans, large metadata, expensive filter predicates, and per-recipient synchronous logging can dominate fanout. More partitions are not a free speedup; they add tables and coordination and should be chosen from representative contention and membership shape. Optimize only after preserving the wrapper’s semantic invariants.
8. Extensions & Challenges
8.1 Beginner Extensions
- Add a diagnostic command listing memberships for the current process.
- Add a subscriber class filter with a fixed vocabulary.
- Compare explicit unsubscribe with process exit cleanup.
8.2 Intermediate Extensions
- Add Registry listener observations and explain why crash notification still requires monitoring when needed.
- Compare Registry membership with a central GenServer under churn.
- Add property-based tests for repeated subscribe/unsubscribe sequences.
8.3 Advanced Extensions
- Evaluate key-oriented versus PID-oriented duplicate partition strategies on an Elixir version that supports explicit selection.
- Add a custom dispatcher that batches sends while preserving the bounded caller-path rule.
- Design, but do not yet implement, the adapter boundary needed to propagate one publication to other nodes.
9. Real-World Connections
9.1 Industry Applications
- Local cache invalidation: Several cache processes can observe one node-local key family.
- Plugin notifications: Process-owned plugin workers subscribe to capability topics.
- Device gateways: Local handlers join region or device-class topics.
- Instrumentation hooks: Diagnostic processes attach temporarily and disappear cleanly on exit.
- Phoenix.PubSub internals: Local subscription storage and fanout use Registry concepts before an adapter crosses nodes.
9.2 Related Open Source Projects
- Elixir — Registry implementation and documentation.
- Phoenix.PubSub — production framework abstraction over local fanout plus adapters.
- Horde — distributed process registry and supervision, useful for contrast rather than a drop-in Pub/Sub answer.
9.3 Interview Relevance
You can now compare state ownership, contention, lifecycle cleanup, and failure semantics rather than simply saying “Registry is faster.” You should be able to explain why a local decentralized index helps, where dispatch work executes, why duplicate mode needs an application contract, and why none of this creates durable or cross-node delivery.
10. Resources
10.1 Essential Reading
- Elixir Registry — complete reference for duplicate keys and dispatch.
- Elixir Supervisor — infrastructure ordering and restart strategy.
- Elixir Process — process ownership and inspection.
- “Enterprise Integration Patterns” — Ch. 3 Publish-Subscribe Channel.
- “Designing for Scalability with Erlang/OTP” — process architecture and supervision.
10.2 Video and Talk Topics
- ElixirConf talks on Registry, ETS contention, and application supervision.
- Erlang Solutions talks comparing process registries and distributed registries.
- Production case studies of high-cardinality local routing.
10.3 Tools and Documentation
- IEx attached sessions: Run several interactive subscriber processes on one node.
- ExUnit: Assert messages, process exit, and cleanup.
- Registry inspection API: Diagnose keys, counts, and lookups.
- Benchee, optional: Profile representative functions after correctness; do not make it part of the core dependency set.
10.4 Related Projects in This Series
- Previous: Project 1 — Mailbox Broadcast Laboratory established the send and mailbox mechanics used here.
- Next: Project 3 — Contract-Aware Domain Event Hub adds domain envelopes, compatibility, revision gaps, and authorized topic derivation.
- Project 4 — Phoenix.PubSub Notification Hub replaces this custom local bus with the framework abstraction.
11. Self-Assessment Checklist
11.1 Understanding
- I can distinguish unique and duplicate Registry keys.
- I can explain process-owned membership cleanup.
- I know why raw duplicate registration may produce duplicate delivery.
- I can explain how partitions affect callback assumptions.
- I can identify work that must not happen inside dispatch.
- I can explain why Registry is node-local and volatile.
11.2 Implementation
- Registry starts under supervision before subscribers.
- Application code uses only the wrapper.
- Repeated logical subscribe is idempotent.
- Metadata is validated and bounded.
- Dispatch merges callback batches correctly.
- Dead processes leave no ghost memberships.
- Diagnostics distinguish routing from handling.
- Tests cover no-subscriber and exit races.
11.3 Growth
- I can choose between a GenServer map and Registry for a concrete use case.
- I can explain one case where a central owner is preferable.
- I measured subscription churn as well as publication.
- I documented Registry’s non-guarantees.
12. Submission / Completion Criteria
Minimum Viable Completion
- A supervised duplicate Registry holds one topic with two subscriber PIDs.
- Publishing reaches both.
- Subscriber exit removes its membership.
Full Completion
- The wrapper enforces idempotent logical subscription.
- Topic and metadata validation are explicit.
- Partition-safe dispatch performs only bounded routing work.
- Diagnostics show entries, sends, partitions, and duration.
- Churn and lifecycle tests prove no ghost memberships.
- Documentation clearly states node-local, volatile, no replay, and no processing acknowledgement.
Excellence
- A representative churn/fanout report compares configurations without overclaiming.
- Property-based subscription sequences preserve membership invariants.
- The design notes identify the exact adapter seam that Project 4 will replace.
Return to the Pub/Sub in Elixir mastery guide.
See the expanded project index.