Project 54: Measure-Theoretic Probability Sandbox

A self-contained deep-dive from the canonical Math from Foundations to Machine Learning curriculum.

  • Difficulty: Level 4: Expert
  • Time: 2 weeks
  • Language: Python
  • Prerequisites: Projects 12, 14, 26, 29, and 52-53
  • Source: ML-Math P26

What You Will Build

Construct a finite-world sandbox where formal probability objects are executable and inspectable. Represent finite sample spaces, event families, generated sigma-algebras, and probability measures; validate empty/full events, complements, unions, normalization, non-negativity, and additivity. Let users define random variables and target measurable sets, then test measurability through preimages. Compute distributions and expectations both from atoms and induced values, and add experiments illustrating almost-sure, in-probability, and expectation behavior without claiming finite simulations prove infinite theorems.

Real World Outcome

Given a finite coin tree or dice experiment, the program prints the generated sigma-algebra, measure-validation results, preimage evidence for measurability, induced distribution, and matching expectation calculations. Invalid event collections and non-additive weights produce precise counterexamples showing the failed axiom.

Core Question

What structural assumptions make probability statements, random variables, and expectations mathematically valid?

Concepts You Must Understand First

  • Sigma-algebras and probability measures — Patrick Billingsley, Probability and Measure.
  • Generated event systems, closure, atoms, and finite additivity/countable-additivity intuition.
  • Measurable functions through preimages of measurable sets.
  • Expectation as a Lebesgue integral, reduced to weighted sums in finite spaces.
  • Almost-sure, in-probability, and mean convergence as distinct modes.

Build Milestones

  1. Represent finite sample spaces/events and generate closure under complement and union.
  2. Validate sigma-algebra axioms and probability-measure normalization/additivity with counterexamples.
  3. Implement random variables, induced distributions, and explicit preimage measurability checks.
  4. Compute expectation from sample atoms and grouped output values; require agreement.
  5. Add convergence-mode demonstrations and a report connecting assumptions to ML probability claims.

Hints in Layers

  1. Use immutable sets (frozenset) so events can themselves be members of collections.
  2. In finite spaces, generate a sigma-algebra iteratively until no new complements/unions appear.
  3. Start measurability with every subset of a finite codomain, then try a coarser target sigma-algebra.

Common Pitfalls and Debugging

  • Any collection of events is accepted: closure was not checked to a fixed point. Return the missing complement or union as a witness.
  • Random variable is assumed measurable: its preimages were never tested. Enumerate target measurable sets and report the first invalid preimage.
  • Two expectation methods disagree: probabilities were not grouped correctly or measure additivity failed. Validate the measure first and print per-value contributions.

Definition of Done

  • Sigma-algebra generation and validation work on several finite spaces.
  • Invalid structures return a concrete failed axiom and witness events.
  • Measurability is decided through explicit preimage tests.
  • Atom-based and induced-distribution expectations agree within tolerance.
  • At least two convergence modes are contrasted with honest simulation limitations.
  • A short report connects formal assumptions to one ML convergence statement.

Previous: Project 53 · Complete learning path · Next: Project 55