Project 7: Interactive SLA Command Center

Build a reactive Livebook dashboard whose KPIs, latency distributions, breach trends, and drilldowns remain truthful under rapid filter changes, late results, empty populations, stale data, and partial failures.

Quick Reference

Attribute	Value
Main language	Elixir
Alternative languages	SQL; optional JavaScript for visualization comparison
Primary tools	Livebook, Kino, Explorer, VegaLite, KinoVegaLite
Difficulty	Level 3 — Advanced
Coolness	Level 4 — Seriously Cool
Business potential	Level 3 — Strong internal product
Estimated time	18–28 hours
Observable deliverables	Interactive dashboard, generation-safe update loop, state matrix, soak-test report
Main learning focus	Reactive correctness and honest operational metrics

1. Learning Objectives

By the end of this project, you will be able to:

Define availability, breach rate, and latency percentiles with explicit populations, units, and time windows.
Choose visual encodings that match operational questions rather than merely decorating a notebook.
Model interactive controls as an event stream with request generations and clear ownership of mutable UI state.
Prevent a slow older computation from overwriting a newer user request.
Bound chart and table payloads before they reach the browser.
Distinguish initial, loading, ready, empty, partial, stale, and failed states.
Preserve independent state for multiple connected clients where the design requires it.
Verify dashboard behavior with deterministic race tests and a sustained interaction soak test.

The command center is successful when operators can tell what the metrics mean and whether they should trust the current view. A fast chart with an ambiguous denominator is a failure.

2. Theoretical Foundation

2.1 Core Concepts

Metric contracts and denominators. “Availability” is not self-defining. You must specify the eligible request population, success criteria, excluded synthetic traffic, treatment of client errors, time zone, window boundaries, and aggregation method. “SLA breaches” can mean breached requests, breached time buckets, or violated objectives; each has a different denominator. Every KPI needs a label, unit, population, time window, freshness rule, and null/empty behavior.

Percentiles and distributions. The p95 latency is the value at or below which roughly 95% of the selected observations fall under a declared quantile convention. It does not describe the worst 5%, and averaging per-host p95 values usually does not produce the global p95. Percentiles must be computed from a compatible population or from a mergeable sketch with understood error. A histogram reveals shape that p50/p95/p99 cards cannot: multiple modes, a long tail, clipping, and sparse bins.

Event-driven UI state. Kino controls emit events. A robust dashboard normalizes each control state into a request, assigns a monotonically increasing generation, computes a result, and updates a frame only if that generation is still current. One process or clearly defined owner should serialize authority over a client’s current generation and render state.

Late-result suppression. Computations can finish out of order. Suppose generation 17 is slow and generation 18 is fast. If both can write directly to the same frame, generation 17 may overwrite the correct newer view. Cancellation can save work, but suppression is still necessary because cancellation may race with completion or may not interrupt every backend operation.

Bounded rendering. A browser should receive aggregated chart points and a small drilldown, not millions of requests. Bound the number of series, bins, time buckets, table rows, text bytes, and retained historical frames. Aggregation belongs before rendering.

2.2 Why This Matters

Operational dashboards influence incident decisions. A view that silently treats an empty population as 100% availability can end an investigation too early. A stale dashboard without a freshness label can direct responders toward a condition that no longer exists. A late computation that overwrites a new filter can make an operator believe they are viewing one service while actually seeing another.

Livebook and Kino make it easy to add interactive controls and frames, while Explorer and VegaLite make aggregation and visualization expressive. That ease raises the standard: the notebook must handle asynchronous behavior, data semantics, and UI state deliberately. The project therefore treats reactivity as a correctness problem.

The patterns transfer to admin consoles, observability tools, business dashboards, model-evaluation interfaces, and any system where users change inputs faster than computations reliably finish.

2.3 Historical Context / Background

Static operational reports evolved into periodically refreshed dashboards, then into highly interactive observability systems. Each step reduced time-to-question but introduced state and concurrency problems. A static report has one snapshot; an interactive dashboard has user intent, in-flight work, cached data, client-specific state, and asynchronous completion order.

Reactive notebook libraries bridge exploratory analysis and lightweight applications. Kino controls provide input events, while Kino frames provide updateable output areas. Vega-Lite introduced a declarative grammar in which data fields map to visual channels such as position, color, and size. Declarative charts reduce imperative drawing code, but they cannot choose a truthful metric definition or bound the data for you.

Production observability platforms often preaggregate metrics, maintain sketches for quantiles, and encode freshness and missingness. This project recreates those design pressures at a manageable scale inside Livebook.

2.4 Common Misconceptions

“The last computation to finish is the latest request.” Completion order and request order are independent.
“Cancellation alone prevents stale updates.” A cancelled task can race with completion; generation checks remain necessary.
“No rows means zero breaches and therefore perfect availability.” No eligible observations means the KPI is undefined or empty, not automatically perfect.
“p99 is the slowest request.” It summarizes a quantile, not the maximum.
“Averaging p95 values gives the global p95.” Quantiles are generally not algebraically composable that way.
“More points make a chart more accurate.” Excess browser payload can make interaction unusable without adding decision value.
“A shared frame naturally gives every client independent state.” State scope and ownership must be designed explicitly.
“Freshness is just another timestamp.” It is a contract relating source observation time, ingestion completion, and the current wall clock.

3. Project Specification

3.1 What You Will Build

Create an interactive SLA command center for a synthetic multi-service request dataset. The dashboard must include:

A source strip showing dataset snapshot, event-time coverage, last refresh, current data age, and request generation.
Filters for period, service, severity, and region.
KPI cards for request count, availability, SLA-breach rate, and p50/p95/p99 latency.
A bounded latency histogram or distribution view.
A breach trend chart with explicit time-bucket and denominator semantics.
A bounded drilldown table containing only safe, useful operational fields.
An update loop that discards obsolete generations.
Explicit UI variants for initial, loading, ready, empty, partial, stale, and failed states.
A deterministic slow-query simulation that proves generation 17 cannot overwrite generation 18.
Evidence that two clients have independent filters and generations where intended.

Use Explorer for transformation and aggregation, Kino controls for interaction, Kino frames for updates, and VegaLite/KinoVegaLite for declarative charts. A static fixture is sufficient; the difficulty lies in state truthfulness, not data acquisition.

3.2 Functional Requirements

Source and filters

Display logical source identity, snapshot/version, coverage interval, refresh time, and freshness status.
Normalize all filter states into a validated request.
Assign a monotonically increasing generation to each accepted request.
Reject invalid or excessively broad combinations before expensive work.

Metric contracts

Define the request population and success/breach predicates.
Display units and selected window on every KPI group.
Treat missing and empty populations explicitly.
Compute percentiles from the selected population using one documented method.

Charts and drilldown

Aggregate into bounded time buckets and histogram bins before rendering.
Limit series count and group rare categories deliberately.
Limit drilldown rows and indicate truncation.
Preserve consistent axes or clearly signal when axes change between filters.

Reactive correctness

Render LOADING immediately for the new current generation.
Allow an old task to finish but discard its update if its generation is no longer current.
Represent each terminal state with generation, source freshness, and status evidence.
Avoid unbounded task, frame, event-log, and chart-data accumulation.

3.3 Non-Functional Requirements

A typical filter request should produce bounded chart datasets and no more than the configured drilldown rows.
Browser responsiveness must remain acceptable during a 30-minute interaction soak.
Every frame update must be attributable to a request generation.
Failure in one component must not be mislabeled as a fully ready dashboard.
A partial dashboard must name which components failed or are stale.
Metric definitions must remain accessible from the dashboard itself.
No raw secrets or sensitive payload bodies may be shown in the drilldown.
A clean runtime replay must reproduce fixture-based KPI expectations.

3.4 Example Usage / Output

The ready state should include a compact evidence panel like:

SERVICE: checkout-api       WINDOW: last 24 hours
Requests: 8,241,993         Availability: 99.94%
SLA breaches: 0.37%         p50 / p95 / p99: 42 / 211 / 804 ms
Data age: 47 seconds        Status: CURRENT
Filter generation: 18
Late result generation 17: discarded

The empty state must not render perfect service:

SERVICE: training-unused-service   WINDOW: last 15 minutes
Eligible requests: 0
Availability: N/A
Latency percentiles: N/A
Status: EMPTY_POPULATION
Data age: 31 seconds

A partial state may resemble:

Generation: 22
KPI cards: READY
Breach trend: READY
Latency histogram: FAILED (invalid bucket contract)
Drilldown: TRUNCATED at 100 rows
Overall status: PARTIAL

3.5 Real World Outcome

An operator can filter service behavior, understand what each KPI measures, and trust that the view corresponds to the latest accepted request. Rapid filter changes never show an older result as current. Empty traffic, stale sources, partial component failures, and hard failures are visibly distinct.

A reviewer can inspect the generation transcript and see:

when each request was accepted;
when loading began;
which computation finished first;
why an old completion was discarded;
how much data was aggregated and rendered;
which freshness and metric contracts governed the result.

The outcome is a defensible lightweight operational interface and a reusable concurrency pattern for Livebook applications.

4. Solution Architecture

4.1 High-Level Design

 +------------------------ Client session -------------------------+
 | period | service | severity | region controls                   |
 +------------------------------+----------------------------------+
                                | control events
                                v
 +-----------------------------------------------------------------+
 | Request owner                                                   |
 | normalize -> validate -> generation := generation + 1           |
 | current_request := {client_id, generation, filters}             |
 +------------------+--------------------------+-------------------+
                    |                          |
                    | immediate                | async computation
                    v                          v
 +--------------------------+       +-------------------------------+
 | Frame: LOADING gen N     |       | Data source / Explorer plan   |
 | previous view not current|       | filter -> aggregate -> bound  |
 +--------------------------+       +---------------+---------------+
                                                   |
                                                   v
                                      +-----------------------------+
                                      | Result tagged {client, gen} |
                                      +--------------+--------------+
                                                     |
                                      +--------------v--------------+
                                      | Is gen still current?       |
                                      +-------+---------------------+
                                              |
                               no: discard <--+--> yes: classify state
                                                        |
                               +------------------------+----------------+
                               |                        |                |
                               v                        v                v
                       +--------------+         +---------------+ +------------+
                       | KPI cards    |         | Vega-Lite data| | Drilldown  |
                       | definitions  |         | bounded points| | capped rows|
                       +--------------+         +---------------+ +------------+
                                                        |
                                                        v
                                           Frame: READY/EMPTY/PARTIAL/
                                                  STALE/FAILED gen N

4.2 Key Components

Component	Responsibility	Key invariant
Source adapter	Return fixture snapshot and freshness metadata	Data and metadata describe the same snapshot
Control group	Emit a coherent filter request	No half-updated filter combinations
Request owner	Own current generation per client	Only owner decides whether a result is current
Compute worker	Filter and aggregate data	Every result is tagged with client and generation
Metric engine	Apply declared KPI contracts	Numerator and denominator are observable
Payload limiter	Bound bins, buckets, series, and rows	Browser never receives unbounded raw events
State classifier	Map evidence to UI state	Empty, stale, partial, and failure stay distinct
Frame renderer	Update one coherent dashboard surface	Update accepted only for current generation
Event transcript	Record transitions and discarded results	Retention is bounded

4.3 Data Structures

FilterRequest
  client_id
  generation
  accepted_at_monotonic
  window_start_utc, window_end_utc
  service, severity, region
  max_drilldown_rows

MetricContract
  id, version
  label, unit
  eligible_population predicate
  numerator and denominator definitions
  empty behavior
  freshness threshold

DashboardResult
  client_id, generation
  source_snapshot, source_age_seconds
  status: ready | empty | partial | stale | failed
  KPI values plus population counts
  bounded histogram bins
  bounded trend buckets
  bounded drilldown rows and truncation flag
  component errors

ClientState
  current_generation
  current_filters
  active_task identity (optional)
  bounded transition log

Do not store the raw full dataset inside DashboardResult. It is a render contract, not a cache of every source event.

4.4 Algorithm Overview

on coherent control change:
  validate and normalize filters
  increment this client's generation
  render LOADING for the new generation
  optionally request cancellation of older task
  start bounded computation tagged with client and generation

on computation completion:
  compare result generation to current client generation
  if older: record DISCARDED and do not update frame
  if current:
    classify freshness and component outcomes
    render one coherent terminal state

For n selected events, filtering and exact aggregation are typically O(n). Exact percentile computation may require O(n log n) if sorting is used, while selection algorithms can improve expected complexity; the chosen method must be documented. Histogram and time-bucket updates are O(n + b) for b bounded output buckets. Render cost must depend on bounded output size, not the raw n.

5. Implementation Guide

5.1 Development Environment Setup

Use a current Livebook release with mutually compatible Kino, Explorer, VegaLite, and KinoVegaLite packages. Record resolved versions in the notebook because package versions change.

Create a deterministic request-event fixture spanning multiple services, regions, severities, and time windows.
Include known empty filter combinations, a stale snapshot, a component-error fixture, and a latency tail.
Add a controllable delay map so one filter request can be forced to finish after a later request.
Verify Kino controls emit events and a Kino frame can be updated without accumulating a new output per event.
Verify a small Vega-Lite chart renders from a bounded aggregate.

Setup evidence:

fixture_snapshot: sla-training-v1
known_request_count: recorded
known_empty_filter: available
delay_injection: available
frame_update_smoke_test: PASS
bounded_chart_smoke_test: PASS

5.2 Project Structure

— Dashboard contract and operator questions
— Runtime and package versions
— Fixture source, snapshot, and freshness policy
— Metric definitions and expected fixture truth
— Static aggregation proof
— Vega-Lite encoding prototypes
— Controls and coherent request normalization
— Per-client request owner and generations
— Asynchronous bounded computation
— Late-result suppression
— State classifier
— Frame composition
— Drilldown and payload bounds
— Race, empty, stale, partial, and failure tests
— Multi-client and 30-minute soak evidence

Build the static metric pipeline before adding events. Reactivity should orchestrate a known-correct analytical function, not conceal uncertain metric logic.

5.3 The Core Question You’re Answering

How do you keep an analytical view truthful while users and data change faster than computation?

Your answer must cover semantic truth (metric contract, population, freshness) and temporal truth (the rendered generation is the newest accepted request). Both are necessary.

5.4 Concepts You Must Understand First

Before implementation, explain:

The numerator, denominator, window, and empty behavior for availability and breach rate.
The difference between p99 and maximum latency.
Why global percentiles cannot usually be reconstructed by averaging subgroup percentiles.
How a control event becomes an owned request generation.
Why late-result checks are required even when cancellation exists.
How many points and rows each dashboard component may render.
How stale and partial differ from failed.

5.5 Questions to Guide Your Design

Is the time window based on event time, ingestion time, or processing time?
Which HTTP or domain-specific outcomes count as available?
Are maintenance windows excluded, and how is that visible?
What happens to requests with missing latency or unknown service identity?
Are filters applied atomically as one coherent request?
Which process owns the current generation for each client?
Can two clients affect one another’s selected filters or frame?
What is cancelled, and what is merely ignored when obsolete?
How do you cap series when a region or service dimension has high cardinality?
How do you indicate chart truncation or grouped “other” values?
When source age crosses its threshold during a displayed result, does the state transition to stale without a new filter event?

5.6 Thinking Exercise

Draw a timeline for this sequence:

t0: generation 17 accepted; forced computation delay = 900 ms
t1: generation 18 accepted; computation delay = 100 ms
t2: generation 18 finishes
t3: generation 17 finishes

For every time, record the owner state, active tasks, frame status, and transcript entry. Then add a client B whose generation numbering is independent. Finally, make generation 18 return zero eligible rows and decide which KPIs are N/A, zero, or still meaningful.

5.7 The Interview Questions They’ll Ask

How do you prevent stale asynchronous work from overwriting current UI state?
What is the difference between cancellation and late-result suppression?
How would you define and validate an availability metric?
Why are percentiles difficult to aggregate across partitions?
How do you distinguish an empty result from a failed data source?
What limits would you place on an interactive operational dashboard?

5.8 Hints in Layers

Hint 1 — Compute statically first. Prove KPIs and fixture expectations with no controls.

Hint 2 — Use one coherent request. Combine the current control values, validate them, then allocate a generation.

Hint 3 — Tag everything. Worker result, frame state, transcript event, and error must carry client and generation.

Hint 4 — Accept updates at one gate. Only the state owner compares a result with the current generation and authorizes rendering.

Hint 5 — Bound before encoding. The Vega-Lite dataset should already contain only intended bins or buckets.

Hint 6 — Empty is a state. Keep population count and KPI nullability in the result contract.

Pseudocode sketch:

request = normalize_controls(current_values)
generation = owner.accept(request)
frame.render(loading(generation))
worker.start(request, generation)

when result arrives:
  if owner.is_current?(result.client_id, result.generation):
    frame.render(classify_and_present(result))
  else:
    transcript.record(discarded(result.generation))

5.9 Books That Will Help

Topic	Book	Suggested focus
Visual encodings	Fundamentals of Data Visualization — Claus O. Wilke	Scales, distributions, uncertainty
Dashboard design	The Big Book of Dashboards — Wexler, Shaffer, Cotgreave	Choosing views for operational questions
Concurrency and state	Elixir in Action — Saša Jurić	Processes, message passing, state ownership
Reliability metrics	Site Reliability Engineering — Beyer et al.	SLOs, monitoring, alerting
Streaming systems	Designing Data-Intensive Applications — Martin Kleppmann	Event time, stream processing, derived views

5.10 Implementation Phases

Phase 1 — Metric and fixture contract (3–4 hours)

Define source coverage, freshness, eligible population, KPI formulas, and empty behavior.
Create fixture truth for ready, empty, stale, and tail-latency cases.
Validate static results.

Phase 2 — Static dashboard and bounded encodings (3–5 hours)

Design KPI cards, histogram, trend, and drilldown.
Choose explicit Vega-Lite field types, units, bins, and scales.
Prove chart and table payload bounds.

Phase 3 — Event ownership and generations (4–6 hours)

Combine controls into coherent requests.
Add per-client generation ownership and loading state.
Tag asynchronous results and suppress obsolete completions.

Phase 4 — Complete state model (3–5 hours)

Implement initial, loading, ready, empty, partial, stale, and failed rendering.
Surface component failures, truncation, and freshness consistently.
Bound transition history.

Phase 5 — Concurrency and soak verification (3–5 hours)

Run deterministic generation 17/18 inversion.
Verify two-client isolation.
Run a 30-minute rapid-interaction soak and inspect process, memory, frame, and payload behavior.

5.11 Key Implementation Decisions

Decision	Options	Required evidence
State scope	Global, per notebook, per client	Two-client behavior matches intent
Obsolete work	Cancel, ignore, both	Old result never overwrites current frame
Percentiles	Exact, approximate sketch	Accuracy and population contract documented
Empty KPI	`N/A`, zero, omit	No false perfect-availability signal
Freshness	One threshold, metric-specific thresholds	Age and threshold displayed
Series cap	Top N, allowlist, grouped other	Truncation/grouping visible
Drilldown cap	Fixed count or byte budget	Partial indicator visible
Error policy	Whole-dashboard failure or component partial	State reflects actual usable evidence

6. Testing Strategy

6.1 Test Categories

Test the metric contract, asynchronous generation ordering, complete UI state model, bounded rendering, and multi-client isolation separately. Their combination proves that the dashboard is numerically meaningful and remains correct under interaction and concurrency.

6.2 Critical Test Cases

Metric tests

Hand-calculate a tiny fixture and compare request count, availability, breach rate, p50, p95, and p99.
Verify window boundaries use one explicit inclusive/exclusive convention.
Verify missing latency and excluded traffic follow the declared population rules.
Verify an empty eligible population produces N/A where the denominator is zero.
Verify the same selected population feeds KPI, histogram, trend, and drilldown.

Race and generation tests

Force generation 17 to take 900 ms.
Accept generation 18 after 50 ms and let it finish in 100 ms.
Assert generation 18 renders ready.
Let generation 17 finish.
Assert the frame remains generation 18 and transcript records 17 as discarded.
Repeat around cancellation timing boundaries.

State matrix

Condition	Expected state	Required evidence
No request yet	INITIAL	Instructions and source identity
Current request running	LOADING	Generation and filters
Complete current data	READY	KPIs, age, population
Zero eligible observations	EMPTY	Successful query plus zero population
Some components usable	PARTIAL	Component success/failure list
Source age over threshold	STALE	Age, threshold, last refresh
Source/computation unusable	FAILED	Safe error classification and request ID

Boundedness tests

Generate many services and verify series cap/grouping.
Generate dense events and verify fixed histogram bins and trend buckets.
Generate more drilldown matches than allowed and verify truncation indicator.
Verify transition log retention remains fixed under thousands of events.
Inspect browser payload and runtime memory over the soak test.

Multi-client tests

Client A and B choose different services.
Each receives independent current generation and result.
A’s late result cannot overwrite B’s frame or vice versa.
Disconnect one client and verify its tasks/state are cleaned up according to policy.

6.3 Test Data

Keep a hand-calculated fixture for exact KPI assertions, deterministic slow and fast generations for race tests, an empty population, stale timestamps, component failures, high-cardinality services, dense events, and two named client sessions. Seed every fixture so failures can be replayed byte-for-byte.

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

Problem: Old data flashes after rapid filter changes

Why: workers update a shared frame directly or results lack generations.
Evidence: rendered generation decreases or filter label and data disagree.
Fix: route all completions through one owner and accept only the current client generation.
Quick test: deterministic slow generation 17 followed by fast generation 18.

Problem: Empty traffic displays 100% availability

Why: a zero denominator was coerced to a success ratio.
Evidence: eligible request count is zero while availability is numeric.
Fix: encode empty behavior in the metric contract and render N/A with EMPTY state.

Problem: Browser slows during a long session

Why: raw points, unbounded table rows, retained frames, event logs, or orphan tasks accumulate.
Evidence: payload size, process count, or memory grows with every interaction.
Fix: aggregate before rendering, update one frame, cap logs, and clean obsolete/client-disconnected work.

Problem: p95 changes unexpectedly when grouped

Why: subgroup percentiles were averaged or populations differ between views.
Evidence: exact reference calculation disagrees with grouped calculation.
Fix: compute from the selected population or use a documented mergeable quantile sketch.

Problem: Dashboard shows ready with one broken chart

Why: component failures are swallowed or overall status is hard-coded.
Evidence: missing visualization with no component error list.
Fix: classify per-component outcomes and derive PARTIAL when usable evidence remains.

Problem: Two users change each other’s filters

Why: controls, generation, or frame are globally shared despite per-client intent.
Evidence: one client’s event changes another client’s rendered service.
Fix: key state by client/session and define cleanup; or explicitly label a deliberately shared dashboard.

7.2 Debugging Strategies

Trace one client request through normalized filters, allocated generation, worker start, completion, acceptance or discard, and frame update. Always display the rendered generation and population contract in diagnostic mode. Reproduce races with deterministic delays before reaching for timing-dependent logging.

7.3 Performance Traps

Raw event rendering, unbounded series, growing transition histories, orphan workers, repeated full-data scans, and large frame payloads can exhaust either runtime or browser. Measure both server memory and encoded browser payload, cap retained state, and verify obsolete generations cannot continue expensive work indefinitely.

8. Extensions & Challenges

8.1 Beginner Extensions

Approximate quantiles: compare exact p99 with a mergeable sketch and report error bounds.
SLO burn rate: add short- and long-window burn-rate panels with precise error-budget semantics.

8.2 Intermediate Extensions

Streaming fixture: append synthetic events and transition from current to stale when updates stop.
Cross-filter brushing: select a histogram region and update the drilldown while preserving generations.
Snapshot URL/state export: serialize safe filter state for reproducible incident handoff.

8.3 Advanced Extensions

Accessibility pass: keyboard controls, non-color status cues, table alternatives, and readable chart descriptions.
Performance budgets: automatically fail a verification cell when payload, latency, or memory bounds regress.
Independent computation service: move metric execution behind a bounded API while preserving the same result/state contract.

Each extension must retain deterministic race tests and the full state matrix.

9. Real-World Connections

9.1 Industry Applications

SRE and observability: availability, tail latency, freshness, and error budgets guide incident response.
Business intelligence: filters and charts need coherent populations and explicit empty states.
Reactive applications: generation tokens are a general solution to out-of-order async completion.
Distributed systems: event time, ingestion time, partial failure, and stale replicas shape what “current” means.
Frontend performance: bounded payloads and retained-state limits matter even when computation occurs server-side.
Multi-user systems: state ownership and isolation must match collaboration semantics.

Kino, VegaLite, and Explorer form the core notebook stack. Prometheus and OpenTelemetry provide useful comparisons for metric and telemetry semantics, while Grafana illustrates production dashboard concerns such as shared definitions and alert links. Compare contracts and failure states rather than attempting feature parity.

9.3 Interview Relevance

A production command center would add authenticated access, centralized metric definitions, durable query services, caching, rate limits, tracing, alert links, audit logging, and deployment health. The Livebook version isolates the fundamental contracts before that platform work.

10. Resources

10.1 Essential Reading

Site Reliability Engineering edited by Betsy Beyer and others — monitoring distributed systems and service objectives.
Designing Data-Intensive Applications by Martin Kleppmann — time, streams, state, and distributed failure.

10.2 Video Resources

Use current Livebook/Kino demonstrations and SRE conference talks to study interaction and metric communication. Validate every percentile and state transition against this project’s deterministic fixtures.

10.3 Tools & Documentation

Official documentation

Reliability and visualization

Verify current package compatibility and API details when starting the project; the conceptual contracts are more durable than version numbers.

11. Self-Assessment Checklist

11.1 Understanding

I can state every KPI’s population, numerator, denominator, unit, window, and empty behavior.
I can explain the percentile method and why subgroup p95 values are not simply averaged.
I can trace a control event through request normalization, generation, computation, and rendering.
I can prove generation 17 cannot overwrite generation 18.
I can distinguish cancellation from late-result suppression.

11.2 Implementation

I can demonstrate initial, loading, ready, empty, partial, stale, and failed states.
I can show exact bounds for series, bins, buckets, rows, log entries, and payload size.
I can prove two clients behave according to the declared state scope.
I can explain which source timestamp determines freshness.
I can run the dashboard for 30 minutes without unbounded process, memory, or browser growth.

11.3 Growth

I can add one interactive view while preserving generation ordering and payload limits.
I can explain what must change before the notebook becomes a production command center.

12. Submission / Completion Criteria

Submit the Livebook notebook, deterministic fixture, metric-contract table, state-transition diagram, race-test transcript, boundedness measurements, and 30-minute soak report.

The submission is complete when:

This expansion belongs to Livebook + Elixir Deep Dive. Return to the expanded project index for the rest of the sprint.