Project 15: Actor Model Framework

Quick Reference

Attribute Value
Project Number 15 of 17
Difficulty Master
Time Estimate 4-6 weeks
Language C++ (C++17 minimum, C++20 recommended)
Alternative Languages Erlang (for comparison), Rust, Scala
Prerequisites Projects 3 (Thread Pool) and 6 (Lock-Free Queue) completed
Knowledge Area Concurrency Patterns / Message Passing
Key Concepts Actor isolation, mailboxes, supervision trees, message dispatch, location transparency
Main Book “C++ Concurrency in Action, Second Edition” by Anthony Williams
Secondary Reference “Reactive Design Patterns” by Roland Kuhn
Coolness Level Level 5: Pure Magic (Super Cool)
Business Potential Open Core Infrastructure

Learning Objectives

By completing this project, you will:

  1. Master the Actor Concurrency Model: Understand how isolating state within actors eliminates data races by design
  2. Build Lock-Free Mailboxes: Implement per-actor message queues using lock-free techniques from Project 6
  3. Design a Message Dispatch System: Create type-safe message handling using std::variant or std::any with pattern matching
  4. Integrate with Thread Pool Schedulers: Execute actors efficiently on your work-stealing thread pool from Project 3
  5. Implement Supervision Hierarchies: Build fault-tolerant systems where parent actors manage child lifecycle and failures
  6. Create Ask/Tell Communication Patterns: Implement both fire-and-forget and request-reply messaging with futures
  7. Understand Location Transparency: Abstract away whether actors run locally or remotely (foundation for distributed systems)
  8. Build Reactive Systems: Learn how message-driven architectures enable responsive, resilient, and elastic systems

Theoretical Foundation

Core Concepts

The Actor Model: A Different Way to Think About Concurrency

The Actor Model, introduced by Carl Hewitt in 1973, flips the traditional approach to concurrent programming on its head:

Traditional Concurrency (Shared State):

    Thread 1        Thread 2        Thread 3
       |              |               |
       v              v               v
    +--------------------------------------+
    |          SHARED MEMORY               |
    |  +---------+  +---------+            |
    |  | Counter |  |  Queue  |  ......    |
    |  +---------+  +---------+            |
    +--------------------------------------+
          |              |
          v              v
    [MUTEX]  [MUTEX]  [MUTEX]  ...

    Problems:
    - Deadlocks (lock ordering)
    - Data races (forgotten locks)
    - Complex reasoning (any thread can touch any data)

Actor Model (Message Passing):

    +----------+        +----------+        +----------+
    | Actor A  |        | Actor B  |        | Actor C  |
    |          |        |          |        |          |
    | [state]  |        | [state]  |        | [state]  |
    | [mailbox]|<--msg--|          |--msg-->| [mailbox]|
    |          |        | [mailbox]|<--msg--|          |
    +----------+        +----------+        +----------+
         |                   |                   |
         v                   v                   v
    Process messages one at a time (no concurrent access!)

    Benefits:
    - No shared mutable state between actors
    - No locks needed (messages are copied/moved)
    - Each actor processes one message at a time
    - Easy to reason about (sequential within actor)

Key insight: In the actor model, concurrency exists between actors (they run in parallel), but within each actor, execution is sequential. This eliminates the need for synchronization primitives like mutexes within actor logic.

The Three Axioms of Actors

When an actor receives a message, it can do exactly three things:

  1. Send messages to other actors it knows about (including itself)
  2. Create new actors (becoming their supervisor)
  3. Designate behavior for the next message (change internal state)
Message Received by Actor:

    +------------------+
    | Incoming Message |
    +--------+---------+
             |
             v
    +------------------+
    |    Actor Logic   |
    +--------+---------+
             |
    +--------+---------+---------+
    |        |         |         |
    v        v         v         v
  Send    Create     Change    Reply
Messages  Actors     State   (optional)

These three axioms are complete. Any concurrent computation can be expressed using only these operations.

Actor Lifecycle and Mailboxes

Every actor has:

  • Identity (ActorRef): A handle for sending messages
  • State: Private data that only this actor can access
  • Behavior: Code that processes incoming messages
  • Mailbox: A queue of pending messages
Actor Lifecycle:

    Created
       |
       v
    +---------+    receive()    +-----------+
    | Idle    |--------------->| Processing |
    | (in     |<---------------| Message    |
    | mailbox)|    done         +-----------+
    +---------+                      |
       |                             | error
       | stop()                      v
       v                      +-------------+
    +---------+               | Failed      |
    | Stopped |               | (notify     |
    +---------+               | supervisor) |
                              +-------------+
                                     |
                                     v
                              Supervisor decision:
                              - Restart
                              - Stop
                              - Escalate

Message Types and Dispatch

Messages in an actor system need type erasure since a mailbox must hold arbitrary message types:

Message Dispatch Options:

Option 1: std::variant (compile-time, type-safe)

    using Message = std::variant<
        Increment,
        GetCount,
        Reset,
        Shutdown
    >;

    void receive(const Message& msg) {
        std::visit(overloaded{
            [this](const Increment& m) { count += m.amount; },
            [this](const GetCount& m) { reply(Count{count}); },
            [this](const Reset&) { count = 0; },
            [this](const Shutdown&) { stop(); }
        }, msg);
    }

    Pros: Fast dispatch, exhaustive matching
    Cons: All message types must be known at compile time

Option 2: std::any (runtime, flexible)

    void receive(const std::any& msg) {
        if (auto* inc = std::any_cast<Increment>(&msg)) {
            count += inc->amount;
        } else if (auto* get = std::any_cast<GetCount>(&msg)) {
            reply(Count{count});
        }
        // ...
    }

    Pros: Add message types at runtime
    Cons: Slower dispatch, no compile-time checking

Option 3: Virtual base class

    struct Message { virtual ~Message() = default; };
    struct Increment : Message { int amount; };

    void receive(std::unique_ptr<Message> msg) {
        if (auto* inc = dynamic_cast<Increment*>(msg.get())) {
            count += inc->amount;
        }
        // ...
    }

    Pros: Familiar OOP pattern
    Cons: Heap allocation per message, RTTI overhead

Supervision Trees: Fault Tolerance by Design

In Erlang/OTP (and Akka), actors are organized into supervision trees. When a child actor fails, its parent (supervisor) decides what to do:

Supervision Hierarchy:

                    +------------------+
                    |   /user          |  (system root)
                    | (UserGuardian)   |
                    +--------+---------+
                             |
            +----------------+----------------+
            |                |                |
    +-------v------+  +------v-------+  +-----v--------+
    | /user/app    |  | /user/db     |  | /user/web    |
    | (AppSupervisor)|  | (DBSupervisor)|  | (WebSupervisor)|
    +-------+------+  +------+-------+  +------+-------+
            |                |                 |
    +-------+-------+   +----+----+      +-----+-----+
    |       |       |   |         |      |           |
   Actor   Actor  Actor Conn1   Conn2   Handler1  Handler2
    A       B       C   Pool    Pool    Worker    Worker

Supervision Strategies:

1. One-for-One:
   Child fails --> Only that child is restarted

2. One-for-All:
   Child fails --> All children are restarted

3. Rest-for-One:
   Child fails --> That child and all children after it are restarted

Why supervision matters: Traditional exception handling assumes the caller can fix the problem. But in concurrent systems, errors often occur in components you don’t directly call. Supervision provides a structured way to handle these failures.

Ask vs Tell Patterns

Two fundamental communication patterns exist between actors:

Tell Pattern (Fire and Forget):

    actorRef.tell(MyMessage{data});  // Returns immediately
    // No response expected
    // Sender continues without waiting

    Use when:
    - Commands that don't need confirmation
    - Events being broadcast
    - High-throughput scenarios


Ask Pattern (Request-Reply):

    std::future<Response> future = actorRef.ask<Response>(Query{});
    // Returns a future immediately
    // Can await the response later

    Response response = future.get();  // Blocks until response

    Use when:
    - Need response to continue
    - Implementing synchronous APIs over actors
    - Gathering data from multiple actors


Implementation of Ask:

    +----------+                    +----------+
    | Sender   |                    | Receiver |
    +----------+                    +----------+
         |                               |
         | 1. Create temp actor          |
         | 2. Send message with          |
         |    replyTo = temp actor       |
         |------------------------------>|
         |                               |
         |       3. Process message      |
         |                               |
         |    4. Send reply to replyTo   |
         |<------------------------------|
         |                               |
         | 5. Temp actor receives,       |
         |    fulfills promise           |
         |                               |
         v                               v
    future.get() returns with response

Why This Matters

Industry Adoption: The actor model powers some of the most demanding systems:

Company System Scale
WhatsApp Erlang-based messaging 2 billion users, 65 billion messages/day
LinkedIn Akka for feed processing 500M members, real-time updates
Discord Elixir/Erlang infrastructure 19M concurrent users peak
Microsoft Orleans .NET Virtual Actors Xbox Cloud Gaming, Halo
Twitter Finagle (actor-like) 500M tweets/day

Career Impact: Understanding actors prepares you for:

  • Distributed systems design (actors naturally span machines)
  • Game engine development (Unity’s DOTS is actor-inspired)
  • Financial systems (event sourcing + actors)
  • IoT and embedded systems (message-driven is natural for hardware)

Mental Model Benefits: Even when not using actors directly, thinking in terms of isolated components communicating via messages improves:

  • Microservices design
  • Event-driven architectures
  • Reactive programming patterns

Historical Context

1973: Carl Hewitt publishes “A Universal Modular ACTOR Formalism for Artificial Intelligence” at MIT, introducing the actor model as a mathematical theory of computation.

1986: Erlang is developed at Ericsson for telephone switches. It implements actors (called “processes”) with supervision trees. Erlang systems achieve “nine nines” reliability (99.9999999% uptime).

2009: Akka framework brings actors to the JVM (Scala/Java), heavily inspired by Erlang/OTP.

2012: Microsoft introduces Orleans, a “Virtual Actor” model that automatically manages actor lifecycle and distribution.

2015: Elixir gains popularity, bringing modern syntax to the Erlang VM with full actor support.

2020s: Actors influence modern patterns:

  • Swift Actors (language-level support in Swift 5.5)
  • Rust’s actix framework
  • C++ proposals for executor-based actors

Common Misconceptions

Misconception Reality
“Actors are just threads” Actors are logical entities scheduled on thread pools. Millions of actors can run on a handful of threads.
“Actors are slow because of message copying” Move semantics eliminate most copies. Lock-free mailboxes provide sub-microsecond latency.
“You need Erlang/Scala for actors” Actors are a pattern, not a language feature. C++ implementations can be very efficient.
“Actor systems are always distributed” Actors work great for single-machine concurrency. Distribution is an extension.
“Supervision means automatic error recovery” Supervision provides structure for recovery. You still design the recovery strategy.
“Actors eliminate all concurrency bugs” Actors eliminate data races but not logical race conditions (wrong message order, timeout bugs).

Project Specification

What You Will Build

A complete actor framework for C++ with the following components:

  1. Actor Base Class: Foundation for user-defined actors with message handling
  2. ActorRef: Type-erased handle for sending messages to actors
  3. Mailbox: Lock-free per-actor message queue using techniques from Project 6
  4. ActorSystem: Container managing actor lifecycle and scheduling on thread pool
  5. Supervision: Parent-child relationships with restart policies
  6. Ask/Tell Patterns: Fire-and-forget and request-reply messaging

Functional Requirements

Core Actor Operations

// Define a custom actor
class CounterActor : public Actor {
    int count = 0;

public:
    // Message handlers
    void receive(const Increment& msg) {
        count += msg.amount;
    }

    void receive(const GetCount& msg) {
        reply(Count{count});
    }

    void receive(const Reset&) {
        count = 0;
    }
};

// Create actor system
ActorSystem system(8);  // 8 worker threads

// Spawn actors
auto counter = system.spawn<CounterActor>("counter");

// Tell (fire and forget)
counter.tell(Increment{5});
counter.tell(Increment{3});

// Ask (request-reply)
auto future = counter.ask<Count>(GetCount{});
Count result = future.get();
assert(result.value == 8);

Supervision

class WorkerSupervisor : public Actor {
public:
    void preStart() override {
        // Spawn children with supervision
        for (int i = 0; i < 4; ++i) {
            auto worker = spawn<WorkerActor>("worker-" + std::to_string(i));
            supervise(worker, SupervisionStrategy::Restart);
        }
    }

    void onChildFailed(ActorRef child, std::exception_ptr error) override {
        log("Child {} failed, applying supervision strategy", child.path());
        // Default behavior: apply configured strategy
    }
};

Non-Functional Requirements

Requirement Target
Message Throughput > 1M messages/second (single actor receiving)
Actor Creation > 100K actors/second
Ask Latency < 10us average round-trip
Memory per Actor < 1KB (excluding user state)
Scalability Linear with cores up to 16 threads
Thread Safety Zero data races (TSan clean)

Example Usage / Output

// Example: Ping-Pong Actors
struct Ping { int count; ActorRef sender; };
struct Pong { int count; };

class PingActor : public Actor {
public:
    void receive(const Pong& msg) {
        if (msg.count > 0) {
            ponger.tell(Ping{msg.count - 1, self()});
        } else {
            system().shutdown();
        }
    }

    void setTarget(ActorRef target) { ponger = target; }
    void start(int n) { ponger.tell(Ping{n, self()}); }

private:
    ActorRef ponger;
};

class PongActor : public Actor {
public:
    void receive(const Ping& msg) {
        msg.sender.tell(Pong{msg.count});
    }
};

Expected Output:

$ ./actor_demo

Actor System starting with 8 workers...
[System] Spawned SupervisorActor@/user/supervisor
[SupervisorActor] Spawning CounterActor
[SupervisorActor] Spawning WorkerActor
[System] Actor hierarchy:
  /user
    /user/supervisor
      /user/supervisor/counter
      /user/supervisor/worker

[WorkerActor] Processing jobs...
[CounterActor] Increment received, count=1
[CounterActor] Increment received, count=2
[WorkerActor] Error: simulated failure!
[SupervisorActor] Child WorkerActor failed, restarting...
[WorkerActor] Restarted, resuming work...
[CounterActor] GetCount query, responding with 2

Final count: 2

System Statistics:
  Actors created: 3
  Actors currently active: 3
  Messages sent: 1,247
  Messages processed: 1,247
  Average message latency: 1.2us
  System uptime: 5.0s

$ ./actor_benchmark

Benchmark: Message Throughput
  Single producer, single consumer actor
  10,000,000 messages

  Results:
    Total time: 8.3 seconds
    Throughput: 1,204,819 messages/second
    Average latency: 0.83us

Benchmark: Actor Creation
  Creating and stopping actors

  Results:
    10,000 actors created in 89ms
    Creation rate: 112,359 actors/second
    Memory per actor: 896 bytes

Benchmark: Ask Pattern
  Request-reply round trips

  Results:
    100,000 ask operations
    Average round-trip: 4.2us
    P99 round-trip: 12.1us

Real World Outcome

After completing this project, you will have:

  1. A working actor framework that you can use for concurrent applications
  2. Deep understanding of message-passing concurrency vs shared-state concurrency
  3. Experience with lock-free data structures in a real context
  4. Foundation for building distributed systems (actors naturally distribute)
  5. Portfolio piece demonstrating advanced concurrency skills

Solution Architecture

High-Level Design

+------------------------------------------------------------------------+
|                           ActorSystem                                    |
+------------------------------------------------------------------------+
|                                                                          |
|  +---------------------------+     +---------------------------+         |
|  |      Actor Registry       |     |       Thread Pool         |         |
|  |  (name -> ActorCell)      |     |   (from Project 3)        |         |
|  +---------------------------+     +---------------------------+         |
|              |                               |                           |
|              v                               v                           |
|  +--------------------------------------------------------------------+ |
|  |                        Actor Scheduling                             | |
|  |  When actor has messages:                                           | |
|  |    1. Mark as scheduled                                             | |
|  |    2. Submit to thread pool                                         | |
|  |    3. Worker runs actor.processMailbox()                            | |
|  +--------------------------------------------------------------------+ |
|                                                                          |
|  +--------------------------------------------------------------------+ |
|  |                         ActorCell (one per actor)                   | |
|  | +----------------+  +----------------+  +------------------------+  | |
|  | | Actor Instance |  |    Mailbox     |  |    Supervision State   |  | |
|  | | (user code)    |  | (lock-free Q)  |  | - parent ref           |  | |
|  | |                |  |                |  | - child refs           |  | |
|  | | - state        |  | +------------+ |  | - restart policy       |  | |
|  | | - receive()    |  | | Message 1  | |  |                        |  | |
|  | |                |  | | Message 2  | |  |                        |  | |
|  | |                |  | | Message 3  | |  |                        |  | |
|  | +----------------+  | +------------+ |  +------------------------+  | |
|  |                     +----------------+                              | |
|  +--------------------------------------------------------------------+ |
|                                                                          |
+------------------------------------------------------------------------+

Message Flow:

    sender.tell(msg)
         |
         v
    +------------------+
    | Get target's     |
    | ActorCell        |
    +--------+---------+
             |
             v
    +------------------+
    | mailbox.enqueue  |
    | (lock-free push) |
    +--------+---------+
             |
             | if (!scheduled.exchange(true))
             v
    +------------------+
    | threadPool.submit|
    | (processMailbox) |
    +--------+---------+
             |
    [... worker thread picks up ...]
             |
             v
    +------------------+
    | while (msg = mailbox.pop()) |
    |   actor.dispatch(msg)       |
    +------------------+
             |
             v
    scheduled.store(false)
    if (mailbox.hasMore()) reschedule

Key Components

1. Actor Base Class

class Actor {
public:
    virtual ~Actor() = default;

    // Lifecycle hooks (override in subclass)
    virtual void preStart() {}
    virtual void postStop() {}
    virtual void preRestart(std::exception_ptr reason) { postStop(); }
    virtual void postRestart(std::exception_ptr reason) { preStart(); }

    // Supervision hooks
    virtual void onChildFailed(ActorRef child, std::exception_ptr error) {
        // Default: apply supervisor strategy
    }

protected:
    // Available to subclasses
    ActorRef self() const { return self_; }
    ActorRef parent() const { return parent_; }
    ActorSystem& system() const { return *system_; }

    // Spawning children
    template<typename A, typename... Args>
    ActorRef spawn(std::string name, Args&&... args);

    // Supervision
    void supervise(ActorRef child, SupervisionStrategy strategy);

    // Replying to ask() requests
    template<typename T>
    void reply(T&& response);

    // Stop self or children
    void stop();
    void stop(ActorRef child);

private:
    friend class ActorCell;
    friend class ActorSystem;

    ActorRef self_;
    ActorRef parent_;
    ActorSystem* system_ = nullptr;
    ActorRef currentSender_;  // Set during message processing
};

2. ActorRef (Handle)

class ActorRef {
public:
    ActorRef() = default;  // Null ref

    // Tell (fire and forget)
    template<typename M>
    void tell(M&& message) const {
        tell(std::forward<M>(message), ActorRef{});
    }

    template<typename M>
    void tell(M&& message, ActorRef sender) const {
        if (cell_) {
            cell_->enqueue(Envelope{
                std::make_any<std::decay_t<M>>(std::forward<M>(message)),
                sender
            });
        }
    }

    // Ask (request-reply)
    template<typename R, typename M>
    std::future<R> ask(M&& message) const {
        auto promise = std::make_shared<std::promise<R>>();
        auto future = promise->get_future();

        // Create temporary actor to receive reply
        auto tempActor = system_->spawnAnonymous<PromiseActor<R>>(promise);
        tell(std::forward<M>(message), tempActor);

        return future;
    }

    // Identity
    std::string path() const;
    bool operator==(const ActorRef& other) const;
    explicit operator bool() const { return cell_ != nullptr; }

private:
    friend class ActorCell;
    friend class ActorSystem;

    std::shared_ptr<ActorCell> cell_;
    ActorSystem* system_ = nullptr;
};

3. Mailbox (Lock-Free Queue)

class Mailbox {
public:
    // Uses lock-free MPSC queue (multiple producers, single consumer)
    // Single consumer because only one thread processes this actor at a time

    void enqueue(Envelope message) {
        queue_.push(std::move(message));
    }

    std::optional<Envelope> tryDequeue() {
        return queue_.pop();
    }

    bool empty() const {
        return queue_.empty();
    }

private:
    // From Project 6, but MPSC variant
    LockFreeMPSCQueue<Envelope> queue_;
};

4. ActorCell (Actor Container)

class ActorCell : public std::enable_shared_from_this<ActorCell> {
public:
    ActorCell(std::unique_ptr<Actor> actor,
              ActorRef parent,
              std::string name,
              ActorSystem* system);

    void enqueue(Envelope message) {
        mailbox_.enqueue(std::move(message));
        scheduleIfNeeded();
    }

    void processMailbox() {
        // Process up to N messages per scheduling
        constexpr int BATCH_SIZE = 100;

        for (int i = 0; i < BATCH_SIZE; ++i) {
            auto msg = mailbox_.tryDequeue();
            if (!msg) break;

            try {
                actor_->currentSender_ = msg->sender;
                dispatch(msg->message);
            } catch (...) {
                handleFailure(std::current_exception());
            }
        }

        scheduled_.store(false, std::memory_order_release);

        // Check if more messages arrived while processing
        if (!mailbox_.empty()) {
            scheduleIfNeeded();
        }
    }

private:
    void scheduleIfNeeded() {
        bool expected = false;
        if (scheduled_.compare_exchange_strong(expected, true,
                                               std::memory_order_acq_rel)) {
            system_->schedule(shared_from_this());
        }
    }

    void dispatch(const std::any& message);
    void handleFailure(std::exception_ptr error);

    std::unique_ptr<Actor> actor_;
    Mailbox mailbox_;
    std::atomic<bool> scheduled_{false};
    std::string name_;
    std::string path_;
    ActorRef parent_;
    std::vector<ActorRef> children_;
    SupervisionStrategy strategy_ = SupervisionStrategy::Restart;
    ActorSystem* system_;
};

5. ActorSystem

class ActorSystem {
public:
    explicit ActorSystem(size_t numThreads = std::thread::hardware_concurrency())
        : threadPool_(numThreads) {}

    ~ActorSystem() {
        shutdown();
    }

    // Spawn top-level actor
    template<typename A, typename... Args>
    ActorRef spawn(std::string name, Args&&... args) {
        return spawnChild<A>(userGuardian_, std::move(name),
                            std::forward<Args>(args)...);
    }

    // Look up actor by path
    ActorRef lookup(const std::string& path);

    // Graceful shutdown
    void shutdown();

    // Wait for completion
    void awaitTermination();

private:
    friend class ActorCell;
    friend class Actor;

    template<typename A, typename... Args>
    ActorRef spawnChild(ActorRef parent, std::string name, Args&&... args) {
        auto actor = std::make_unique<A>(std::forward<Args>(args)...);
        auto cell = std::make_shared<ActorCell>(
            std::move(actor), parent, std::move(name), this
        );
        registerActor(cell);
        cell->actor_->preStart();
        return ActorRef{cell, this};
    }

    void schedule(std::shared_ptr<ActorCell> cell) {
        threadPool_.submit([cell] {
            cell->processMailbox();
        });
    }

    void registerActor(std::shared_ptr<ActorCell> cell);
    void unregisterActor(const std::string& path);

    WorkStealingThreadPool threadPool_;  // From Project 3
    std::unordered_map<std::string, std::shared_ptr<ActorCell>> registry_;
    std::mutex registryMutex_;
    ActorRef userGuardian_;  // Root of user actor hierarchy
};

Data Structures

Structure Implementation Purpose
Envelope struct { std::any message; ActorRef sender; } Wrap message with sender info
Mailbox Lock-free MPSC queue Per-actor message buffer
ActorCell Actor + Mailbox + metadata Complete actor state
Registry unordered_map<string, ActorCell*> Path-based lookup
Supervision Parent-child links + policy enum Failure handling

Algorithm Overview

Message Sending (Tell)

tell(message, sender):
    1. Look up target actor's cell (O(1) from ActorRef)
    2. Wrap message in Envelope with sender
    3. Enqueue to target's mailbox (lock-free push)
    4. If actor not scheduled:
         a. CAS scheduled flag from false to true
         b. If successful, submit to thread pool

Message Processing

processMailbox():
    for i = 0 to BATCH_SIZE:
        msg = mailbox.tryPop()
        if msg is null: break

        try:
            set currentSender to msg.sender
            dispatch msg to appropriate receive() method
        catch exception:
            notify parent supervisor
            apply supervision strategy

    scheduled = false (release)

    if mailbox not empty:
        reschedule self

Supervision on Failure

handleFailure(exception):
    1. Stop current message processing
    2. Notify parent: parent.onChildFailed(self, exception)
    3. Parent applies strategy:
       - Restart: preRestart(), create new instance, postRestart()
       - Stop: postStop(), remove from hierarchy
       - Escalate: parent throws, its parent handles
    4. If Restart: drain mailbox or process pending messages

Implementation Guide

Development Environment Setup

Required Compiler and Tools

# Ubuntu/Debian
sudo apt install g++-11 cmake ninja-build

# macOS
brew install gcc@11 cmake ninja

# Verify C++17 support
g++ --version  # Should be 11+ for best C++17/20 support

# Install sanitizers (usually included with modern compilers)
# ThreadSanitizer is critical for testing

CMakeLists.txt

cmake_minimum_required(VERSION 3.16)
project(ActorFramework CXX)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

# Optimization for release
set(CMAKE_CXX_FLAGS_RELEASE "-O3 -DNDEBUG")

# Debug with sanitizers
set(CMAKE_CXX_FLAGS_DEBUG "-g -O1 -fsanitize=thread")

# Library
add_library(actors
    src/actor_system.cpp
    src/actor_cell.cpp
    src/actor_ref.cpp
    src/mailbox.cpp
)

target_include_directories(actors PUBLIC include)

# Benchmarks
add_executable(actor_benchmark
    benchmark/throughput.cpp
    benchmark/latency.cpp
)
target_link_libraries(actor_benchmark actors pthread)

# Tests
enable_testing()
add_executable(actor_tests
    tests/actor_test.cpp
    tests/supervision_test.cpp
    tests/ask_test.cpp
)
target_link_libraries(actor_tests actors pthread)
add_test(NAME actor_tests COMMAND actor_tests)

Project Structure

actor_framework/
  include/
    actor.hpp              # Actor base class
    actor_ref.hpp          # ActorRef handle
    actor_system.hpp       # ActorSystem container
    actor_cell.hpp         # Internal: actor + mailbox
    mailbox.hpp            # Lock-free message queue
    envelope.hpp           # Message wrapper
    supervision.hpp        # Supervision strategies
  src/
    actor_system.cpp       # ActorSystem implementation
    actor_cell.cpp         # ActorCell implementation
    actor_ref.cpp          # ActorRef implementation
    mailbox.cpp            # Mailbox implementation
  tests/
    actor_test.cpp         # Basic actor tests
    supervision_test.cpp   # Failure handling tests
    ask_test.cpp           # Request-reply tests
    stress_test.cpp        # Concurrent load tests
  benchmark/
    throughput.cpp         # Messages per second
    latency.cpp            # Round-trip timing
  examples/
    ping_pong.cpp          # Classic actor example
    counter.cpp            # Simple state example
    supervisor.cpp         # Supervision hierarchy
  CMakeLists.txt
  README.md

Core Question

The Core Question You’re Answering:

How can we structure concurrent programs so that data races are impossible by construction, rather than prevented by careful use of locks?

The actor model answers this by:

  1. Encapsulating state within actors (no shared mutable state)
  2. Communication only via immutable messages (no pointer sharing)
  3. Sequential processing within each actor (no concurrent access to actor state)

Concepts You Must Understand First

Before implementing, verify you can answer these questions:

  1. Why does the actor model eliminate data races?
    • Each actor processes one message at a time
    • State is never shared between actors
    • Messages are copied/moved, not referenced
  2. How does a lock-free queue enable efficient mailboxes?
    • Multiple senders can enqueue concurrently (MPSC)
    • No mutex contention on message passing
    • Bounded memory usage with ring buffer
  3. What is the difference between tell and ask?
    • Tell: sender continues immediately, no response
    • Ask: sender gets a future, waits for response
  4. How do supervision trees handle failures?
    • Parent actors supervise children
    • Failure propagates up the tree
    • Supervisor decides: restart, stop, or escalate
  5. Why use std::any or std::variant for messages?
    • Mailbox must hold arbitrary message types
    • Type erasure allows single queue type
    • Runtime dispatch to appropriate handler

Questions to Guide Your Design

Actor Lifecycle:

  • When is an actor’s state initialized (constructor vs preStart)?
  • How do you handle messages sent during actor restart?
  • When should an actor stop itself vs be stopped by parent?

Mailbox Design:

  • MPSC (multiple producer, single consumer) vs MPMC?
  • Bounded vs unbounded queue?
  • What happens when mailbox is full?

Message Dispatch:

  • How do you route messages to the correct handler?
  • What if a message type has no handler?
  • How do you handle exceptions in handlers?

Scheduling:

  • How many messages should an actor process before yielding?
  • How do you ensure fairness among actors?
  • How do you handle actors with huge mailbox backlogs?

Supervision:

  • Where do you store supervision configuration?
  • How do you restart an actor without losing its ActorRef?
  • How do you propagate errors up the supervision tree?

Thinking Exercise

Before coding, trace through this scenario on paper:

Scenario: Three actors - Counter, Worker, Supervisor

1. Supervisor spawns Counter and Worker as children
2. Worker sends Increment(5) to Counter
3. Worker sends Increment(3) to Counter
4. Worker crashes (throws exception)
5. Supervisor restarts Worker
6. New Worker sends GetCount to Counter
7. Counter replies with Count(8)

Draw:
- The actor hierarchy
- Message flow timeline
- State changes in each actor
- Thread assignments during processing

Questions to answer:

  • Which thread runs each actor?
  • What happens to Worker’s pending messages on crash?
  • How does the new Worker get Counter’s ActorRef?
  • What if Counter crashes while processing Increment?

Hints in Layers

Hint 1: Starting Point (Conceptual Direction)

Start with the simplest possible actor system:

  1. Single-threaded (no thread pool yet)
  2. One message type (use std::function<void()> for simplicity)
  3. No supervision
  4. Synchronous processing (process message immediately on tell)

This gives you a working mental model before adding complexity.

Hint 2: Add Asynchronous Processing

Introduce the mailbox and scheduling:

  1. Add a queue to buffer messages
  2. Process queue in a loop
  3. Add a flag to track if actor is scheduled
  4. Submit to thread pool when new message arrives and not scheduled

Key insight: The scheduling flag is the key to efficiency. Without it, you’d submit to thread pool on every message.

Hint 3: Message Type Erasure

Use std::any for flexibility:

struct Envelope {
    std::any message;
    ActorRef sender;
};

// In actor subclass, provide typed handlers
void receive(const Increment& m) { ... }
void receive(const GetCount& m) { ... }

// Dispatch via type checking
void dispatch(const std::any& msg) {
    if (auto* m = std::any_cast<Increment>(&msg)) {
        receive(*m);
    } else if (auto* m = std::any_cast<GetCount>(&msg)) {
        receive(*m);
    } else {
        // Unknown message type
    }
}

For better performance, consider std::variant if message types are known at compile time.

Hint 4: Implementing Ask Pattern

The ask pattern requires a temporary actor to receive the reply:

template<typename R>
class PromiseActor : public Actor {
    std::shared_ptr<std::promise<R>> promise_;

public:
    PromiseActor(std::shared_ptr<std::promise<R>> p) : promise_(p) {}

    void receive(const R& reply) {
        promise_->set_value(reply);
        stop();  // Self-destruct after receiving reply
    }
};

template<typename R, typename M>
std::future<R> ActorRef::ask(M message) {
    auto promise = std::make_shared<std::promise<R>>();
    auto future = promise->get_future();

    auto tempActor = system_->spawnAnonymous<PromiseActor<R>>(promise);
    tell(std::move(message), tempActor);

    return future;
}

Don’t forget timeout handling! A stuck actor will never respond.

Interview Questions

  1. “Explain the actor model and how it prevents data races.”

    Expected answer: The actor model isolates state within actors. Each actor has a mailbox for incoming messages and processes them one at a time. Since state is never shared between actors (messages are copied, not referenced), and processing is sequential within each actor, data races are structurally impossible. Multiple actors can run concurrently on different threads, but they interact only through message passing.

  2. “How would you implement the ‘ask’ pattern in an actor system?”

    Expected answer: The ask pattern returns a future for request-reply communication. Implementation:

    1. Create a promise/future pair
    2. Spawn a temporary “reply-to” actor that holds the promise
    3. Send the message with reply-to as the sender
    4. When target processes message, it sends reply to sender (the temp actor)
    5. Temp actor fulfills the promise and stops itself
    6. Original caller gets result via future.get()

    Also mention: timeout handling, memory management of temp actor.

  3. “What are the tradeoffs between actors and traditional mutex-based concurrency?”

    Expected answer:

    • Actors: No data races by design, but message passing overhead, potential message ordering issues, learning curve
    • Mutexes: Direct shared access, familiar model, but deadlock risk, composition difficulty
    • Actors excel at: distributed systems, fault tolerance, many independent entities
    • Mutexes excel at: tight coupling, high-frequency shared access, simple critical sections

  4. “How would you design supervision in an actor system?”

    Expected answer: Actors form a hierarchy where each actor supervises its children. When a child fails:

    1. Child catches exception, notifies parent
    2. Parent applies supervision strategy:
      • Restart: Call preRestart, create new instance, call postRestart
      • Stop: Call postStop, remove from hierarchy
      • Escalate: Parent re-throws, its parent handles
    3. Optionally resume mailbox processing or drop pending messages

    Key insight: Separation of concerns - business logic in actor, error handling in supervisor.

  5. “How would you scale an actor system across multiple machines?”

    Expected answer: Location transparency is key. ActorRefs should work the same whether the actor is local or remote:

    1. Serialize messages (Protocol Buffers, FlatBuffers)
    2. ActorRef includes location info (host:port)
    3. Remote send goes through network layer
    4. Remote node deserializes and delivers to local actor
    5. Cluster membership and actor discovery (registry service)
    6. Handle network partitions (timeouts, retries, circuit breakers)

Books That Will Help

Topic Book Relevant Chapters
Actor Model Theory “Reactive Design Patterns” by Roland Kuhn Chapters 1-4 (Actor fundamentals)
Erlang/OTP Patterns “Designing for Scalability with Erlang/OTP” by Cesarini & Vinoski Chapters 3-6 (Supervision)
C++ Concurrency “C++ Concurrency in Action” by Williams Chapter 4 (Message passing)
Lock-Free Queues “C++ Concurrency in Action” by Williams Chapter 7 (Lock-free data structures)
Akka Implementation “Akka in Action” by Roestenburg Chapters 2-5 (Actor patterns)
Distributed Systems “Designing Data-Intensive Applications” by Kleppmann Chapter 8 (Distributed actors)

Implementation Phases

Phase 1: Basic Actors (Week 1)

Goal: Single-threaded actor system with tell().

// Milestone: This should work
class EchoActor : public Actor {
public:
    void receive(const std::string& msg) {
        std::cout << "Echo: " << msg << std::endl;
    }
};

ActorSystem system(1);
auto echo = system.spawn<EchoActor>("echo");
echo.tell(std::string("Hello"));
echo.tell(std::string("World"));
// Output: Echo: Hello\nEcho: World

Checklist:

  • Actor base class with lifecycle hooks
  • ActorRef with tell()
  • Simple mailbox (std::queue + mutex is fine initially)
  • ActorSystem spawns actors
  • Single worker thread processes actors

Phase 2: Lock-Free Mailbox (Week 2)

Goal: Replace mutex-protected queue with lock-free MPSC.

// Reuse or adapt your Project 6 lock-free queue
// Key difference: MPSC (multiple producer, single consumer)
// because each actor has one consumer (the actor itself)

Checklist:

  • Implement or adapt MPSC lock-free queue
  • Replace mutex-protected mailbox
  • Verify no data races with ThreadSanitizer
  • Benchmark: should see throughput improvement

Phase 3: Thread Pool Integration (Week 3)

Goal: Actors run on your work-stealing thread pool.

// Scheduling logic
void ActorCell::enqueue(Envelope msg) {
    mailbox_.push(std::move(msg));

    bool expected = false;
    if (scheduled_.compare_exchange_strong(expected, true)) {
        system_->schedule(shared_from_this());
    }
}

void ActorCell::processMailbox() {
    // Process batch of messages
    for (int i = 0; i < BATCH_SIZE; ++i) {
        auto msg = mailbox_.pop();
        if (!msg) break;
        dispatch(*msg);
    }

    scheduled_.store(false);
    if (!mailbox_.empty()) {
        enqueue({});  // Reschedule if more work
    }
}

Checklist:

  • Integrate with Project 3 thread pool
  • Implement scheduling flag (atomic CAS)
  • Batch processing (N messages per schedule)
  • Benchmark scaling with thread count

Phase 4: Ask Pattern (Week 4)

Goal: Request-reply with futures.

// Milestone: This should work
class Calculator : public Actor {
public:
    void receive(const Add& msg) {
        reply(Result{msg.a + msg.b});
    }
};

auto calc = system.spawn<Calculator>("calc");
auto future = calc.ask<Result>(Add{2, 3});
Result r = future.get();
assert(r.value == 5);

Checklist:

  • PromiseActor for receiving replies
  • ask() method on ActorRef
  • reply() method on Actor
  • Timeout handling for stuck asks

Phase 5: Supervision (Week 5)

Goal: Parent-child relationships with restart policies.

// Milestone: This should work
class Worker : public Actor {
public:
    void receive(const DoWork& msg) {
        if (msg.shouldFail) {
            throw std::runtime_error("Simulated failure");
        }
        // Process work
    }
};

class Supervisor : public Actor {
public:
    void preStart() override {
        auto worker = spawn<Worker>("worker");
        supervise(worker, SupervisionStrategy::Restart);
    }

    void onChildFailed(ActorRef child, std::exception_ptr e) override {
        std::cout << "Child failed, will restart" << std::endl;
    }
};

// Worker fails and is automatically restarted

Checklist:

  • Parent-child tracking in ActorCell
  • SupervisionStrategy enum (Restart, Stop, Escalate)
  • Exception handling in message dispatch
  • preRestart/postRestart hooks
  • Integration tests for failure scenarios

Phase 6: Polish and Optimization (Week 6)

Goals:

  1. Handle edge cases
  2. Add actor naming and lookup
  3. Performance tuning
  4. Documentation

Checklist:

  • Actor path naming (/user/supervisor/worker)
  • Lookup by path
  • Graceful shutdown (stop all actors)
  • Memory leak verification (valgrind)
  • Documentation and examples
  • Comprehensive benchmark suite

Key Implementation Decisions

Decision 1: Message Type Representation

Options:

  • std::any - Runtime flexibility, slower dispatch
  • std::variant - Compile-time safety, faster dispatch
  • Virtual base class - OOP pattern, heap allocation

Recommendation: Start with std::any for flexibility. Optimize to std::variant for hot paths once working.

Decision 2: MPSC vs MPMC Mailbox

Analysis: Each actor processes its own mailbox, so there’s only one consumer. MPSC (multiple producer, single consumer) is simpler and faster than full MPMC.

Recommendation: Use MPSC lock-free queue.

Decision 3: Scheduling Granularity

Options:

  • Process all messages per schedule (unfair)
  • Process one message per schedule (too much overhead)
  • Process N messages per schedule (balance)

Recommendation: Process batch of ~100 messages, then yield. Prevents any single actor from starving others.

Decision 4: Actor Identity

Options:

  • Unique integer ID
  • Hierarchical path string (/user/app/worker-1)
  • UUID

Recommendation: Hierarchical paths (like Akka). Readable, debuggable, natural for supervision hierarchy.

Testing Strategy

Unit Tests

TEST(ActorTest, BasicTell) {
    ActorSystem system(1);
    std::atomic<int> received{0};

    class Counter : public Actor {
        std::atomic<int>& count_;
    public:
        Counter(std::atomic<int>& c) : count_(c) {}
        void receive(const int& n) { count_ += n; }
    };

    auto counter = system.spawn<Counter>("counter", std::ref(received));
    counter.tell(5);
    counter.tell(3);

    // Wait for processing
    std::this_thread::sleep_for(100ms);

    EXPECT_EQ(received.load(), 8);
}

TEST(ActorTest, AskPattern) {
    ActorSystem system(4);

    class Echo : public Actor {
    public:
        void receive(const std::string& msg) {
            reply(msg + "!");
        }
    };

    auto echo = system.spawn<Echo>("echo");
    auto future = echo.ask<std::string>(std::string("Hello"));

    EXPECT_EQ(future.get(), "Hello!");
}

Supervision Tests

TEST(SupervisionTest, RestartOnFailure) {
    ActorSystem system(4);
    std::atomic<int> restartCount{0};

    class FragileWorker : public Actor {
        std::atomic<int>& restarts_;
        int failCountdown_ = 3;
    public:
        FragileWorker(std::atomic<int>& r) : restarts_(r) {}

        void postRestart(std::exception_ptr) override {
            restarts_++;
        }

        void receive(const Ping&) {
            if (--failCountdown_ > 0) {
                throw std::runtime_error("Intentional failure");
            }
            reply(Pong{});
        }
    };

    class Supervisor : public Actor {
        std::atomic<int>& restarts_;
    public:
        Supervisor(std::atomic<int>& r) : restarts_(r) {}

        void preStart() override {
            auto worker = spawn<FragileWorker>("worker", std::ref(restarts_));
            supervise(worker, SupervisionStrategy::Restart);
        }
    };

    auto sup = system.spawn<Supervisor>("supervisor", std::ref(restartCount));
    auto worker = system.lookup("/user/supervisor/worker");

    // Trigger failures and restarts
    worker.tell(Ping{});
    worker.tell(Ping{});
    worker.tell(Ping{});

    std::this_thread::sleep_for(500ms);

    EXPECT_GE(restartCount.load(), 2);
}

Stress Tests

TEST(StressTest, HighThroughput) {
    ActorSystem system(8);
    std::atomic<int64_t> processed{0};
    constexpr int NUM_MESSAGES = 1'000'000;

    class Counter : public Actor {
        std::atomic<int64_t>& count_;
    public:
        Counter(std::atomic<int64_t>& c) : count_(c) {}
        void receive(const int&) { count_++; }
    };

    auto counter = system.spawn<Counter>("counter", std::ref(processed));

    auto start = std::chrono::high_resolution_clock::now();

    // Multiple sender threads
    std::vector<std::thread> senders;
    for (int t = 0; t < 8; ++t) {
        senders.emplace_back([&counter, NUM_MESSAGES]() {
            for (int i = 0; i < NUM_MESSAGES / 8; ++i) {
                counter.tell(1);
            }
        });
    }

    for (auto& t : senders) t.join();

    // Wait for all to be processed
    while (processed.load() < NUM_MESSAGES) {
        std::this_thread::sleep_for(10ms);
    }

    auto end = std::chrono::high_resolution_clock::now();
    auto ms = std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count();

    std::cout << "Throughput: " << (NUM_MESSAGES * 1000 / ms) << " msg/sec\n";
    EXPECT_GT(NUM_MESSAGES * 1000 / ms, 500'000);  // At least 500K/sec
}

Sanitizer Testing

# Thread Sanitizer (critical for actor systems)
cmake -DCMAKE_BUILD_TYPE=Debug ..
make actor_tests
./actor_tests

# Should report zero data races

# Address Sanitizer (memory issues)
cmake -DCMAKE_CXX_FLAGS="-fsanitize=address" ..
make actor_tests
./actor_tests

Common Pitfalls & Debugging

Pitfall 1: Forgetting to Schedule Actor

Problem: Messages are enqueued but never processed.

// WRONG: Enqueue without scheduling
void ActorCell::enqueue(Envelope msg) {
    mailbox_.push(std::move(msg));
    // Missing: schedule the actor!
}

// RIGHT: Schedule if not already scheduled
void ActorCell::enqueue(Envelope msg) {
    mailbox_.push(std::move(msg));
    scheduleIfNeeded();  // Critical!
}

Symptom: Actor appears to ignore messages.

Fix: Always check scheduling flag after enqueue.

Pitfall 2: Race in Scheduling Flag

Problem: Double-scheduling or missed scheduling.

// WRONG: Non-atomic check-then-act
void scheduleIfNeeded() {
    if (!scheduled_) {
        scheduled_ = true;
        system_->schedule(this);
    }
}

// RIGHT: Atomic compare-and-exchange
void scheduleIfNeeded() {
    bool expected = false;
    if (scheduled_.compare_exchange_strong(expected, true,
                                           std::memory_order_acq_rel)) {
        system_->schedule(shared_from_this());
    }
}

Symptom: Spurious duplicate processing or dropped messages.

Fix: Use CAS for the scheduling flag.

Pitfall 3: Deadlock in Ask Pattern

Problem: Calling ask() and blocking on result inside an actor.

// WRONG: Blocks the only thread that could process the reply!
class BadActor : public Actor {
    void receive(const Request& r) {
        auto result = other_.ask<Result>(Query{}).get();  // DEADLOCK
        reply(result);
    }
};

Symptom: System hangs, ask never completes.

Fix: Either:

  1. Use tell() and handle reply in separate message
  2. Use async continuation instead of blocking get()
  3. Never block inside an actor

Pitfall 4: Shared State Between Actors

Problem: Passing pointers in messages, breaking actor isolation.

// WRONG: Shared pointer allows concurrent access!
struct BadMessage {
    std::shared_ptr<std::vector<int>> data;  // SHARED MUTABLE STATE!
};

// RIGHT: Move or copy data, no sharing
struct GoodMessage {
    std::vector<int> data;  // Moved into message, actor owns it
};

Symptom: Data races, inconsistent state.

Fix: Messages should own their data. No shared pointers to mutable data.

Pitfall 5: Actor Self-Reference Cycle

Problem: Actor holds strong reference to itself, preventing cleanup.

// WRONG: Prevents actor destruction
class LeakyActor : public Actor {
    ActorRef selfRef_;  // Strong reference to self!

    void preStart() override {
        selfRef_ = self();  // Creates cycle
    }
};

Symptom: Actors never destroyed, memory leak.

Fix: Don’t store self references. Use self() method when needed.

Pitfall 6: Lost Exception in Fire-and-Forget

Problem: Exception in actor is silently swallowed with tell().

// Exception here goes nowhere with tell()
class FragileActor : public Actor {
    void receive(const Work& w) {
        throw std::runtime_error("Oops");
        // Who sees this error?
    }
};

// Without supervision, error is lost
actor.tell(Work{});

Symptom: Silent failures, inconsistent state.

Fix: Implement proper supervision. Log unhandled exceptions. Consider making supervision mandatory.

Debugging Tips

  1. Add logging: Log actor lifecycle events (spawn, message receive, stop)
  2. Use actor paths: Hierarchical paths make logs readable
  3. ThreadSanitizer: Run early and often, catches races immediately
  4. Message tracing: Log message flow with timestamps and actor paths
  5. Deadlock detection: Add timeout to ask(), log when exceeded

Extensions & Challenges

Extension 1: Typed Actors

Add compile-time type safety to message handling:

template<typename... Messages>
class TypedActor : public Actor {
    // Only accepts Messages... types
    // Compile error if you send wrong type
};

class Counter : public TypedActor<Increment, GetCount, Reset> {
    // Must implement receive() for each type
};

Extension 2: Actor Pools

Implement router actors that distribute work:

// Round-robin distribution
auto pool = system.spawnPool<Worker>(8);  // 8 workers
pool.tell(Work{});  // Goes to next worker in rotation

// Broadcast
pool.broadcast(Shutdown{});  // Goes to all workers

Extension 3: Persistence (Event Sourcing)

Add state persistence for fault tolerance:

class PersistentActor : public Actor {
    void persist(const Event& event) {
        // Save event to journal
        journal_.append(event);
        // Apply event to state
        apply(event);
    }

    void recover() {
        // Replay all events from journal
        for (const auto& event : journal_.read()) {
            apply(event);
        }
    }
};

Extension 4: Remote Actors

Add network transparency:

// Local actor
auto local = system.spawn<Worker>("worker");

// Remote actor (on another machine)
auto remote = system.remote("192.168.1.100:8080")
                   .spawn<Worker>("worker");

// Same API!
local.tell(Work{});
remote.tell(Work{});  // Serializes and sends over network

Extension 5: Backpressure

Handle mailbox overflow gracefully:

enum class OverflowStrategy {
    DropOldest,   // Discard oldest message
    DropNewest,   // Discard new message
    Block,        // Block sender (needs async support)
    Fail          // Throw exception
};

auto actor = system.spawn<Worker>("worker",
    MailboxConfig{
        .capacity = 1000,
        .overflow = OverflowStrategy::DropOldest
    });

Challenge: Implement Virtual Actors

Microsoft Orleans-style virtual actors that are automatically activated/deactivated:

// Virtual actor - always exists conceptually
// Framework activates instance when needed
auto player = grains.get<PlayerGrain>("player-123");
player.tell(Attack{});  // Activates if not in memory

// After idle timeout, actor is deactivated
// State is persisted, reloaded on next access

Real-World Connections

How Akka Does It

Akka (Scala/Java) is the most widely-used actor framework:

Akka Concept Your Implementation
ActorSystem ActorSystem
ActorRef ActorRef
Props Template parameters to spawn()
Dispatcher Thread pool integration
SupervisorStrategy SupervisionStrategy enum
Mailbox Lock-free queue
tell / ! tell()
ask / ? ask()

How Erlang Does It

Erlang’s actor model (processes) is built into the language:

Erlang Feature Your Implementation
spawn(fun) system.spawn<Actor>()
Pid ActorRef
! (send) tell()
receive block receive() method
Supervisor module Supervision hierarchy
gen_server Actor base class
Process mailbox Lock-free queue

Production Systems Using Actors

System Technology Scale
WhatsApp Erlang 2B users, 100B messages/day
Discord Elixir/Erlang 19M concurrent users
LinkedIn Akka 500M members
PayPal Akka Financial transactions
Roblox Custom C++ actors Game servers

Resources

Primary References

Resource Description
“C++ Concurrency in Action” Chapter 4 covers message passing patterns
“Reactive Design Patterns” Comprehensive actor pattern guide
Akka Documentation Production actor framework reference
CAF: C++ Actor Framework Open source C++ actor library

Code to Study

Project What to Learn
CAF Production C++ actor implementation
SObjectizer Alternative C++ actor approach
rotor Lightweight C++ actors
Akka Source Reference implementation (Scala)

Supplemental Reading

Topic Resource
Actor Model Theory Carl Hewitt’s original paper
Erlang OTP “Learn You Some Erlang” (free online)
Lock-free Queues Project 6 of this guide
Thread Pools Project 3 of this guide

Self-Assessment Checklist

Understanding

  • I can explain why the actor model prevents data races
  • I can describe the three axioms of actors
  • I understand the difference between tell and ask patterns
  • I can explain how supervision trees provide fault tolerance
  • I understand why mailboxes should be lock-free

Implementation

  • Actor base class with lifecycle hooks works correctly
  • ActorRef supports tell() with type-erased messages
  • Lock-free mailbox handles concurrent producers
  • Thread pool executes actors with proper scheduling
  • Ask pattern returns futures that resolve with replies
  • Supervision restarts failed child actors

Testing

  • Unit tests cover basic actor operations
  • Supervision tests verify failure handling
  • Stress tests demonstrate high throughput
  • All tests pass under Thread Sanitizer
  • No memory leaks detected by Address Sanitizer

Performance

  • Throughput exceeds 500K messages/second
  • Ask latency under 50 microseconds average
  • Scales linearly up to 8+ threads
  • Memory per actor under 2KB

Extensions (Optional)

  • Typed actors with compile-time message checking
  • Actor pools with routing strategies
  • Backpressure handling for mailbox overflow
  • Actor naming and path-based lookup

Submission / Completion Criteria

Minimum Requirements

Your implementation is complete when:

  1. Core Actor Functionality:
    • Actors can be spawned with custom behavior
    • Messages are delivered via tell()
    • Actors process messages sequentially
    • ActorRefs can be passed in messages
  2. Message Handling:
    • Type-erased message storage (std::any or std::variant)
    • Dispatch to appropriate receive() methods
    • Unknown message types handled gracefully
  3. Concurrency:
    • Lock-free mailbox (MPSC queue)
    • Thread pool execution of actors
    • Proper scheduling (no double-schedule, no missed schedule)
    • No data races (TSan clean)
  4. Ask Pattern:
    • ask() returns std::future
    • reply() sends response to sender
    • Timeout handling for stuck asks
  5. Supervision:
    • Parent-child actor hierarchy
    • Configurable restart policy
    • onChildFailed() notification
    • preRestart/postRestart hooks

Deliverables

actor_framework/
  include/
    actor.hpp              # Actor base class
    actor_ref.hpp          # ActorRef handle
    actor_system.hpp       # ActorSystem container
    actor_cell.hpp         # Internal: actor + mailbox
    mailbox.hpp            # Lock-free message queue
    envelope.hpp           # Message wrapper
    supervision.hpp        # Supervision strategies
  src/
    actor_system.cpp
    actor_cell.cpp
    actor_ref.cpp
    mailbox.cpp
  tests/
    actor_test.cpp
    supervision_test.cpp
    ask_test.cpp
    stress_test.cpp
  benchmark/
    throughput.cpp
    latency.cpp
  examples/
    ping_pong.cpp
    counter.cpp
    supervisor.cpp
  CMakeLists.txt
  README.md

Grading Rubric (Self-Assessment)

Component Points Criteria
Actor lifecycle 15 spawn, preStart, postStop work correctly
Message passing 20 tell() delivers, dispatch routes correctly
Lock-free mailbox 15 MPSC queue, no races
Thread pool integration 15 Proper scheduling, scales with cores
Ask pattern 15 futures work, reply() works
Supervision 10 Restart policy applied on failure
Testing 5 Comprehensive tests, TSan clean
Documentation 5 README, code comments

Total: 100 points

Learning Milestones

Milestone 1: Basic Tell Works

Checkpoint: Single actor receives and processes messages.

What You’ve Proven:

  • Actor base class is correct
  • Message dispatch works
  • Basic mailbox functions

Milestone 2: Multi-Actor Communication

Checkpoint: Multiple actors exchange messages.

What You’ve Proven:

  • ActorRef can be passed in messages
  • Actors can find and message each other
  • No message loss under load

Milestone 3: Thread Pool Integration

Checkpoint: Actors run on thread pool with proper scheduling.

What You’ve Proven:

  • Scheduling flag prevents double-scheduling
  • Lock-free mailbox handles concurrent sends
  • Actors scale across cores

Milestone 4: Ask Pattern Works

Checkpoint: Request-reply with futures completes correctly.

What You’ve Proven:

  • Temporary reply actor works
  • Promise/future integration correct
  • Timeout handling in place

Milestone 5: Supervision Restarts Failed Actors

Checkpoint: Parent actor restarts crashed child.

What You’ve Proven:

  • Exception handling in message dispatch
  • Parent notification works
  • Restart preserves actor identity (same ActorRef)

Estimated completion time: 4-6 weeks of focused effort

This project synthesizes your thread pool and lock-free queue knowledge into a high-level abstraction. The actor model is a powerful mental framework that will influence how you think about concurrent systems, even when not using actors directly.