Project 12: Distribution and Random-Variable Visualizer

A self-contained deep-dive from the canonical Math from Foundations to Machine Learning curriculum.

  • Difficulty: Beginner-Intermediate
  • Time: 10-16 hours
  • Language: Python (alternatives: Julia, R, JavaScript)
  • Prerequisites: Project 11
  • Merged from: ML-Math P13; ML-Found P12

What You Will Build

Build an interactive, seeded explorer for uniform, Bernoulli/binomial, Poisson, exponential, and normal distributions. For each family, accept valid parameters, generate samples, compute empirical moments, and overlay a histogram or mass plot with the theoretical PMF/PDF. Plot the CDF separately and make its probability interpretation explicit. Add a Central Limit Theorem experiment that repeatedly sums or averages samples from a clearly non-normal parent distribution and shows the standardized averages becoming approximately normal as sample size grows.

Real World Outcome

Commands such as normal --mean 0 --std 1 --samples 10000 and binomial --n 20 --p .3 produce reproducible plots and reports comparing theoretical and empirical mean/variance. The CDF view answers interval or threshold questions. A CLT panel compares average distributions for n=1, 2, 10, 30, including standardized axes and a metric or visual reference showing convergence toward a bell curve.

Core Question

How do a random variable’s distribution and parameters determine the shapes, averages, variability, and tail behavior observed in data?

Concepts You Must Understand First

  1. Random variables: functions mapping outcomes to numeric values; discrete and continuous cases differ.
  2. PMF, PDF, and CDF: PMFs assign mass, PDFs assign density, and CDFs give cumulative probability. See DasGupta, Probability for Statistics and Machine Learning, Chapters 2-3.
  3. Expectation and variance: theoretical moments can validate a sampler.
  4. Distribution families: Bernoulli/binomial for successes, Poisson for counts, exponential for waiting, normal for additive noise.
  5. Central Limit Theorem: standardized averages often approach a normal distribution under suitable conditions. See Blitzstein and Hwang, Chapters 3-7.

Build Milestones

  1. Define validated distribution objects exposing sampling, PMF/PDF, CDF, mean, and variance where practical.
  2. Implement deterministic sampling and compare sample moments across growing sample sizes.
  3. Overlay normalized histograms/mass bars with theoretical curves and plot CDFs.
  4. Add interval-probability queries and parameter controls with invalid-range diagnostics.
  5. Build the CLT experiment using several parent distributions and sample-size settings.

Hints in Layers

  1. For continuous distributions, histogram height must use density normalization before comparison with a PDF.
  2. Validate samplers first with mean/variance and quantiles; a pretty histogram can hide systematic errors.
  3. Standardize CLT averages with their theoretical mean and standard error so panels are comparable.

Common Pitfalls and Debugging

  • Symptom: PDF values are interpreted as point probabilities. Cause: density and mass were conflated. Fix: demonstrate probability as area over an interval.
  • Symptom: empirical overlays are consistently shifted. Cause: distribution parameters use a different convention, such as rate versus scale. Fix: name conventions and test theoretical moments.
  • Symptom: CLT plots narrow but do not align. Cause: raw averages were not standardized. Fix: center by the parent mean and scale by sigma/sqrt(n).

Definition of Done

  • At least five common distributions support validated parameters and seeded samples.
  • Empirical means/variances converge toward theoretical references.
  • PMF/PDF and CDF plots are correctly labeled and normalized.
  • Threshold/interval probability queries agree with numerical or known cases.
  • A reproducible CLT experiment compares several sample sizes and parent distributions.

Previous: Project 11 · Complete learning path · Next: Project 13