Learn Statistics: From Scratch to Data-Driven Thinking

Goal: Build a strong, intuitive understanding of statistics from the ground up. You will go from basic data summarization to making informed decisions and predictions, using practical, real-world projects.

Why Learn Statistics?

Statistics is the science of learning from data. It’s the foundation of data science, machine learning, and any field where decisions are made under uncertainty. Most developers have a vague notion of it, but few can wield it confidently.

  • Become a Better Thinker: Learn to spot biases, question assumptions, and separate signal from noise in any context.
  • Unlock Data Science: Statistics is the “why” behind the “how” of machine learning algorithms.
  • Build Smarter Products: Use A/B testing and data analysis to build things your users actually want.
  • Win Arguments with Data: Move from “I think” to “I can show you.”

After completing these projects, you will:

  • Intuitively understand core concepts like probability distributions, confidence intervals, and p-values.
  • Be able to clean, analyze, and visualize datasets with professional tools like Python’s Pandas and Matplotlib.
  • Confidently perform hypothesis tests to make data-driven decisions.
  • Build and interpret simple predictive models using linear regression.
  • Never look at a news headline citing a “study” the same way again.

Core Concept Analysis

The Statistics Learning Ladder

Your journey will follow two main paths: Descriptive Statistics (describing what you see) and Inferential Statistics (making guesses about what you can’t see).

┌─────────────────────────────────────────────────────────────┐
│                       6. REGRESSION                         │
│       (Predicting values, e.g., house price from size)      │
└─────────────────────────────────────────────────────────────┘
                                 ▲
                                 │
┌─────────────────────────────────────────────────────────────┐
│                   5. INFERENTIAL STATISTICS                 │
│      (Hypothesis Testing, A/B Tests, p-values, CI)          │
│       (Is this new drug better than the old one?)           │
└─────────────────────────────────────────────────────────────┘
                                 ▲
                                 │
┌─────────────────────────────────────────────────────────────┐
│                 4. PROBABILITY DISTRIBUTIONS                │
│ (Normal, Binomial, Poisson - Modeling random events)        │
│    (What's the range of likely outcomes for coin flips?)    │
└─────────────────────────────────────────────────────────────┘
                                 ▲
                                 │
┌─────────────────────────────────────────────────────────────┐
│                        3. PROBABILITY                       │
│     (The language of uncertainty, Bayes' Theorem)           │
│         (What are the chances of drawing a red card?)       │
└─────────────────────────────────────────────────────────────┘
                                 ▲
                                 │
┌─────────────────────────────────────────────────────────────┐
│                 1 & 2. DESCRIPTIVE STATISTICS               │
│      (Mean, Median, Mode, Variance, Std Dev, Histograms)    │
│            (What does our data look like?)                  │
└─────────────────────────────────────────────────────────────┘

Key Concepts Explained

1. Descriptive Statistics (The “What”)

These tools summarize a dataset into a few key numbers.

  • Measures of Central Tendency: Where is the “center” of the data?
    • Mean: The average. Sum of all values / number of values. Sensitive to outliers.
    • Median: The middle value when sorted. Robust to outliers.
    • Mode: The most frequent value.
  • Measures of Dispersion (Spread): How spread out is the data?
    • Range: Maximum value - Minimum value. Very simple.
    • Variance (σ²): The average of the squared differences from the Mean. Hard to interpret directly.
    • Standard Deviation (σ): The square root of the variance. Easy to interpret as it’s in the original units of the data. A low SD means data is clustered around the mean.
    • Quartiles & IQR: Divides data into four equal parts. The Interquartile Range (IQR = Q3 - Q1) is the middle 50% of the data.

2. Probability (The “Maybe”)

Probability is a number between 0 (impossible) and 1 (certain) that represents the likelihood of an event.

  • Key idea: If we repeat an experiment many times, the proportion of times an event occurs will approach its probability.
  • Conditional Probability: The probability of event A happening, given that event B has already happened. Written as P(A|B).
  • Bayes’ Theorem: A revolutionary idea that lets you update your beliefs in light of new evidence. It connects P(A|B) with P(B|A). It’s the engine behind medical diagnoses and spam filters.

3. Probability Distributions (The “Shape”)

A distribution describes the probabilities of all possible outcomes.

  • Normal Distribution (The “Bell Curve”): Describes many natural phenomena (heights, blood pressure). Defined by its mean and standard deviation.
  • Binomial Distribution: Describes the number of successes in a fixed number of independent trials (e.g., number of heads in 10 coin flips).
  • Poisson Distribution: Describes the number of events in a fixed interval of time or space, if these events happen at a known average rate (e.g., number of customers arriving at a store in an hour).

4. Inferential Statistics (The “Guess”)

This is where we use data from a small sample to make an educated guess (an inference) about a large population.

  • Central Limit Theorem (CLT): The most important idea in statistics. It states that if you take many large enough samples from any population, the distribution of the sample means will be approximately normal. This is what allows us to make inferences.
  • Confidence Interval (CI): An estimated range of values which is likely to include the true population parameter (e.g., “We are 95% confident that the true average height of all men is between 5’9” and 5’11””).
  • Hypothesis Testing: A formal procedure for checking if your data supports a certain hypothesis.
    • Null Hypothesis (H₀): The default assumption, usually stating “no effect” or “no difference” (e.g., “This new drug has no effect”).
    • Alternative Hypothesis (H₁): The claim you want to prove (e.g., “This new drug reduces recovery time”).
    • p-value: The probability of observing your data (or something more extreme), if the null hypothesis were true. A small p-value (typically < 0.05) suggests that your observation is surprising under the null hypothesis, providing evidence against it.

Project List

These projects are designed to be done in order, building your skills from the ground up using Python, a powerful and beginner-friendly tool for statistics.

Project 1: Personal Data Dashboard

  • File: LEARN_STATISTICS_FROM_SCRATCH.md
  • Main Programming Language: Python
  • Alternative Programming Languages: R, Google Sheets
  • Coolness Level: Level 2: Practical but Forgettable
  • Business Potential: 1. The “Resume Gold”
  • Difficulty: Level 1: Beginner
  • Knowledge Area: Descriptive Statistics / Data Visualization
  • Software or Tool: Python, Pandas, Matplotlib
  • Main Book: Think Stats, 2nd Edition by Allen B. Downey

What you’ll build: A simple script that ingests data you’ve collected about yourself (e.g., hours slept, cups of coffee, pages read per day) and calculates key descriptive statistics, then generates a histogram and a box plot.

Why it teaches statistics: This project makes abstract concepts like mean, median, standard deviation, and quartiles tangible because they describe your own life. You’ll see firsthand how an outlier (a sleepless night) affects the mean but not the median.

Core challenges you’ll face:

  • Collecting and formatting data → maps to the basics of data entry in a CSV file
  • Calculating mean, median, and mode → maps to understanding central tendency
  • Calculating variance and standard deviation → maps to quantifying the “spread” or “consistency” of your habits
  • Creating a histogram and box plot → maps to visualizing the shape and distribution of your data

Key Concepts:

Difficulty: Beginner Time estimate: Weekend Prerequisites: Basic Python (variables, lists, functions).

Real world outcome: You’ll run a Python script and it will output something like this to your console and generate a plot:

Analysis of 'Hours Slept' (30 days):
- Mean: 7.2 hours
- Median: 7.5 hours
- Standard Deviation: 1.1 hours
- Min: 4.0 hours, Max: 9.0 hours

You will also see a histogram showing the distribution of your sleep hours, immediately telling you your most common sleep duration.

Implementation Hints:

  1. Data Collection: For one week, record a daily metric in a simple text file named my_data.csv. 
    date,hours_slept,coffees
2025-12-01,7.5,2
2025-12-02,6.0,3
2025-12-03,8.0,1
...
  2. Setup: Install the necessary Python libraries: pip install pandas matplotlib
  3. Python Script: 
    import pandas as pd
import matplotlib.pyplot as plt

# Read the data
df = pd.read_csv('my_data.csv')

# Select the column to analyze
sleep_data = df['hours_slept']

# Calculate descriptive statistics
mean_sleep = sleep_data.mean()
median_sleep = sleep_data.median()
std_dev_sleep = sleep_data.std()

print("Analysis of 'Hours Slept':")
print(f"- Mean: {mean_sleep:.2f} hours")
# ... print other stats

# Create a histogram
plt.figure() # Create a new figure
plt.hist(sleep_data, bins=5, edgecolor='black')
plt.title("Distribution of Hours Slept")
plt.xlabel("Hours")
plt.ylabel("Frequency")
plt.savefig("sleep_histogram.png") # Save the plot to a file
print("Histogram saved to sleep_histogram.png")

    Questions to guide you:

    • Why might the mean and median be different? What does that tell you?
    • If your standard deviation is high, what does that say about your sleep schedule’s consistency?

Learning milestones:

  1. You successfully load a CSV into a Pandas DataFrame → You’ve learned the basic unit of data analysis in Python.
  2. You calculate the mean, median, and standard deviation → You can summarize any dataset.
  3. You generate a histogram → You can visualize the shape and frequency of your data.
  4. You can explain what the standard deviation means in the context of your own data → You’ve built intuition, not just memorized a formula.

Project 2: Is This Game Rigged? A Loot Box Simulator

  • File: LEARN_STATISTICS_FROM_SCRATCH.md
  • Main Programming Language: Python
  • Alternative Programming Languages: JavaScript, C#
  • Coolness Level: Level 3: Genuinely Clever
  • Business Potential: 1. The “Resume Gold”
  • Difficulty: Level 1: Beginner
  • Knowledge Area: Probability / Monte Carlo Simulation
  • Software or Tool: Python, Numpy
  • Main Book: Think Bayes, 2nd Edition by Allen B. Downey

What you’ll build: A simulator that models opening “loot boxes” from a video game. You’ll run thousands of simulated trials to estimate the real-world cost of obtaining a rare item and visualize the distribution of outcomes.

Why it teaches statistics: This project makes probability concrete. Instead of calculating complex formulas, you’ll discover probabilities through experimentation (a “Monte Carlo method”). You’ll build a deep, intuitive understanding of expected value, probability distributions, and the law of large numbers.

Core challenges you’ll face:

  • Modeling a probabilistic event → maps to using a random number generator to simulate a loot box opening
  • Running thousands of simulations → maps to using loops to repeat an experiment many times
  • Calculating the average outcome (Expected Value) → maps to understanding that the average of many random trials converges on a predictable value
  • Visualizing the distribution of costs → maps to seeing that while the average cost might be $100, many people will pay much more

Key Concepts:

  • Probability: “Think Stats” Ch. 4
  • Monte Carlo Method: A Gentle Introduction to Monte Carlo Simulation
  • Expected Value: Khan Academy - Expected Value
  • The Law of Large Numbers: As you run more trials, the average result gets closer to the expected value.

Difficulty: Beginner Time estimate: Weekend Prerequisites: Project 1, comfort with loops and functions in Python.

Real world outcome: You’ll define the drop rates for items in a loot box (e.g., Legendary: 1%, Epic: 10%, Common: 89%). Your script will then simulate buying boxes until you get a Legendary item, and repeat this 10,000 times. It will output:

Simulating 10,000 players trying to get a Legendary item...
(Loot box cost: $1)

- Average cost: $100.23
- Cheapest attempt: $1
- Most expensive attempt: $750
- 95% of players got it for less than: $300

Distribution of costs has been saved to lootbox_costs.png

The generated histogram will visually prove that a “1% chance” doesn’t mean you’re guaranteed to get it in 100 tries.

Implementation Hints:

  1. Setup: pip install numpy matplotlib
  2. Define Probabilities: 
    # Item: probability
DROP_RATES = {
    'Legendary': 0.01,
    'Epic': 0.10,
    'Common': 0.89
}
LOOT_BOX_COST = 1
  3. Simulate One “Player”: Write a function simulate_one_player() that:
    • Initializes cost = 0.
    • Enters a while True loop.
    • In the loop, cost += LOOT_BOX_COST.
    • Generate a random number between 0 and 1: roll = np.random.rand().
    • Check if you got the legendary: if roll < DROP_RATES['Legendary']: break.
    • The function returns the final cost.
  4. Run Many Simulations:
    • Create an empty list all_costs = [].
    • Loop 10,000 times, calling simulate_one_player() in each iteration and appending the result to all_costs.
  5. Analyze and Visualize:
    • Convert all_costs to a NumPy array for easy calculations: costs_arr = np.array(all_costs).
    • Calculate the mean, max, and percentiles: np.mean(costs_arr), np.max(costs_arr), np.percentile(costs_arr, 95).
    • Create a histogram of costs_arr using Matplotlib.

Learning milestones:

  1. Your script can simulate a single probabilistic event → You understand how to model chance.
  2. You can simulate one player’s entire journey to getting the item → You can model a sequence of random events.
  3. The average cost from your simulation is close to the theoretical expected value (1 / 0.01 = 100) → You’ve witnessed the Law of Large Numbers in action.
  4. You can look at the histogram and explain why some players are “unlucky” and pay much more than the average → You understand the concept of a probability distribution.

Project 3: A “Dumb” Spam Filter

  • File: LEARN_STATISTICS_FROM_SCRATCH.md
  • Main Programming Language: Python
  • Alternative Programming Languages: N/A
  • Coolness Level: Level 3: Genuinely Clever
  • Business Potential: 2. The “Micro-SaaS / Pro Tool”
  • Difficulty: Level 2: Intermediate
  • Knowledge Area: Conditional Probability / Bayes’ Theorem
  • Software or Tool: Python, Pandas, Scikit-learn
  • Main Book: Data Science from Scratch, 2nd Edition by Joel Grus

What you’ll build: A simple spam filter for SMS messages using the Naive Bayes algorithm. You’ll train it on a dataset of real messages labeled as “spam” or “ham” (not spam).

Why it teaches statistics: This is a classic, tangible application of Bayes’ Theorem. You will learn how to “update your beliefs” (the probability of a message being spam) based on the evidence (the words in the message). It’s a bridge from pure statistics to machine learning.

Core challenges you’ll face:

  • Understanding Bayes’ Theorem intuitively → maps to *P(Spam Word) = P(Word Spam) * P(Spam) / P(Word)*
  • Calculating word probabilities → maps to counting word frequencies in spam vs. ham messages
  • Handling words not seen during training → maps to Laplace smoothing (adding 1 to all counts)
  • Combining probabilities for multiple words → maps to the “naive” assumption of independence (and why we use log probabilities)

Key Concepts:

Difficulty: Intermediate Time estimate: 1-2 weeks Prerequisites: Project 1 & 2. Understanding of Python dictionaries.

Real world outcome: You will train your classifier. Then, you can give it a new, unseen message, and it will output a prediction:

New message: "Claim your free prize now! Click here."
Prediction: Spam (Probability: 98.7%)

New message: "Hey, are we still on for lunch tomorrow?"
Prediction: Ham (Probability: 99.9%)

Implementation Hints:

  1. Get Data: Find a dataset. The “SMS Spam Collection Data Set” from the UCI Machine Learning Repository is perfect for this. It’s a single file with two columns: label (ham/spam) and the message text.
  2. Setup: pip install pandas scikit-learn
  3. High-level plan (using Scikit-learn’s tools):
    • Load the data with Pandas.
    • Split the data into a training set and a testing set (train_test_split).
    • Create a data processing “pipeline”:
      • Step 1: Vectorizer (CountVectorizer): This tool converts your text messages into numerical data by counting the occurrences of each word.
      • Step 2: Classifier (MultinomialNB): This is the Naive Bayes algorithm.
    • Train the pipeline on your training data (pipeline.fit()).
    • Test its accuracy on your unseen test data (pipeline.score()).
  4. The “From Scratch” Intuition (what Scikit-learn is doing for you):
    • Calculate Prior Probabilities: What’s the overall probability of any given message being spam? P(Spam) = (Number of spam messages) / (Total number of messages).
    • Calculate Word Probabilities (Likelihoods):
      • For each word in your vocabulary, calculate P(Word | Spam) (How often does this word appear in spam messages?) and P(Word | Ham).
      • This involves counting total words in spam vs. ham and applying Laplace smoothing.
    • Classify a New Message:
      • For a new message, you want to calculate P(Spam | Message).
      • Using Bayes’ rule, this is proportional to P(Message | Spam) * P(Spam).
      • The “naive” part is assuming words are independent: P(Message | Spam) ≈ P(word1 | Spam) * P(word2 | Spam) * ...
      • To avoid numbers getting too small (“underflow”), you add the log probabilities instead of multiplying.
      • Compare the final “score” for spam vs. ham and pick the higher one.

Learning milestones:

  1. You can calculate the prior probability of spam from the dataset → You understand base rates.
  2. You can calculate P("free" | Spam) and P("free" | Ham) → You understand likelihoods and how they form evidence.
  3. Your model correctly classifies most messages in the test set → You’ve successfully built a predictive model.
  4. You can explain why the model classified a message as spam by looking at the word probabilities → You understand how the algorithm “thinks”.

Project 4: Is This Die Loaded?

  • File: LEARN_STATISTICS_FROM_SCRATCH.md
  • Main Programming Language: Python
  • Alternative Programming Languages: R
  • Coolness Level: Level 3: Genuinely Clever
  • Business Potential: 1. The “Resume Gold”
  • Difficulty: Level 2: Intermediate
  • Knowledge Area: Inferential Statistics / Hypothesis Testing
  • Software or Tool: Python, SciPy
  • Main Book: Introductory Statistics (OpenStax, free online textbook)

What you’ll build: A script that uses the Chi-Squared Goodness of Fit test to determine if a series of dice rolls deviates significantly from what you’d expect from a fair die.

Why it teaches statistics: This is a perfect introduction to hypothesis testing. You will formalize a question (“Is this die fair?”), state your null and alternative hypotheses, and use a statistical test to get a p-value that helps you make a conclusion.

Core challenges you’ll face:

  • Formulating a null hypothesis → maps to stating the default assumption: “The die is fair”
  • Calculating expected frequencies → maps to understanding that for a fair die, each side should appear N/6 times in N rolls
  • Understanding the Chi-Squared statistic → maps to quantifying the total difference between observed and expected counts
  • Interpreting the p-value → maps to answering: “If the die was fair, how likely is it we’d see a deviation this large or larger?”

Key Concepts:

  • Hypothesis Testing: “Introductory Statistics” Ch. 9 - OpenStax
  • Chi-Squared Goodness of Fit Test: “Introductory Statistics” Ch. 11
  • p-value: A Gentle Introduction to p-values

Difficulty: Intermediate Time estimate: Weekend Prerequisites: Project 1 & 2.

Real world outcome: You will simulate rolling a die 600 times. First, a fair die, then a loaded die. Your script will analyze the results and produce a clear conclusion.

Analyzing 600 rolls of a simulated FAIR die...
Observed counts: [98, 105, 95, 102, 99, 101]
Expected counts: [100, 100, 100, 100, 100, 100]
Chi-Squared Statistic: 0.56, p-value: 0.989
Conclusion: The p-value is high. We do not have evidence to reject the null hypothesis. The die appears to be fair.

Analyzing 600 rolls of a simulated LOADED die (6 is twice as likely)...
Observed counts: [85, 89, 81, 88, 82, 175]
Expected counts: [100, 100, 100, 100, 100, 100]
Chi-Squared Statistic: 82.1, p-value: 1.5e-16
Conclusion: The p-value is very small. We reject the null hypothesis. The die is likely loaded.

Implementation Hints:

  1. Setup: pip install numpy scipy matplotlib
  2. Simulate Rolls:
    • Fair die: np.random.randint(1, 7, size=600)
    • Loaded die: Use np.random.choice([1, 2, 3, 4, 5, 6], size=600, p=[...]) where you provide custom probabilities (e.g., p=[1/7, 1/7, 1/7, 1/7, 1/7, 2/7]).
  3. Get Observed Counts: Use NumPy’s np.unique(rolls, return_counts=True) to get the counts for each face.
  4. Get Expected Counts: For 600 rolls of a fair die, the expected count for each face is 600 / 6 = 100.
  5. Perform the Test: The scipy library makes this easy. 
    from scipy.stats import chisquare

observed_counts = [...] # Your counts from step 3
expected_counts = [100, 100, 100, 100, 100, 100]

chi2, p_value = chisquare(f_obs=observed_counts, f_exp=expected_counts)

print(f"Chi-Squared Statistic: {chi2:.2f}, p-value: {p_value:.3e}")

alpha = 0.05 # Significance level
if p_value < alpha:
    print("Conclusion: The p-value is small. We reject the null hypothesis.")
else:
    print("Conclusion: The p-value is high. We fail to reject the null hypothesis.")

    The Chi-Squared statistic itself is calculated as Σ [ (Observed - Expected)² / Expected ] for all categories. You can calculate this manually to prove to yourself you understand the formula.

Learning milestones:

  1. You can state a clear Null and Alternative hypothesis for a problem → You understand the foundation of hypothesis testing.
  2. You can calculate the expected frequencies for a given scenario → You can define what “fairness” or “no effect” looks like.
  3. Your script produces a small p-value for the loaded die and a large one for the fair die → You understand what the test output signifies.
  4. You can correctly interpret the p-value in a sentence → You can translate statistical results into a plain English conclusion.

Project 5: Does More Studying Mean Higher Grades?

  • File: LEARN_STATISTICS_FROM_SCRATCH.md
  • Main Programming Language: Python
  • Alternative Programming Languages: R, Excel
  • Coolness Level: Level 2: Practical but Forgettable
  • Business Potential: 1. The “Resume Gold”
  • Difficulty: Level 3: Advanced
  • Knowledge Area: Correlation / Linear Regression
  • Software or Tool: Python, Pandas, Scikit-learn, Matplotlib
  • Main Book: The Art of Data Science by Roger D. Peng and Elizabeth Matsui

What you’ll build: A script that analyzes a dataset of students’ hours studied and exam scores. It will calculate the correlation, fit a linear regression model, and create a scatter plot with the regression line overlaid.

Why it teaches statistics: This is the quintessential introduction to predictive modeling. You will learn to quantify the relationship between two variables and build a simple model that can make predictions (e.g., “If a student studies for 10 hours, what is their predicted score?”).

Core challenges you’ll face:

  • Understanding correlation vs. causation → maps to realizing that just because two variables move together doesn’t mean one causes the other
  • Fitting a line to data → maps to the concept of “least squares,” finding the line that minimizes the total error
  • Interpreting the model’s coefficients → maps to understanding the meaning of the slope and intercept
  • Evaluating model performance → maps to what R-squared means (the proportion of variance explained by the model)

Key Concepts:

  • Correlation: “Introductory Statistics” Ch. 12 - OpenStax
  • Linear Regression: “Introductory Statistics” Ch. 12
  • R-squared: StatQuest: R-squared explained

Difficulty: Advanced Time estimate: 1-2 weeks

  • Prerequisites: All previous projects, a good grasp of high school algebra (the equation of a line, y = mx + b).

Real world outcome: Your script will analyze a dataset and produce a scatter plot. The plot will show a cloud of points (each point is a student) and a straight line running through them. You’ll also get a statistical summary:

Correlation between Hours Studied and Exam Score: 0.89 (Strong positive correlation)

Linear Regression Model:
Score = 5.5 * (Hours Studied) + 45.2

- Intercept (b): 45.2 (Predicted score for 0 hours studied)
- Slope (m): 5.5 (Each additional hour of study is associated with a 5.5 point increase in score)
- R-squared: 0.79 (The model explains 79% of the variance in exam scores)

Prediction for a student who studies 8 hours: 89.2

Implementation Hints:

  1. Data: Create a simple CSV file scores.csv or find one online. 
    hours_studied,exam_score
2,65
3.5,72
...
  2. Setup: pip install pandas scikit-learn matplotlib
  3. Analysis Script: 
    import pandas as pd
from sklearn.linear_model import LinearRegression
import matplotlib.pyplot as plt

# Load and prepare data
df = pd.read_csv('scores.csv')
X = df[['hours_studied']] # Features (needs to be 2D)
y = df['exam_score']      # Target

# Calculate correlation
correlation = df['hours_studied'].corr(df['exam_score'])
print(f"Correlation: {correlation:.2f}")

# Build and train the model
model = LinearRegression()
model.fit(X, y)

# Get model parameters
slope = model.coef_[0]
intercept = model.intercept_
r_squared = model.score(X, y)
print(f"Model: Score = {slope:.2f} * Hours + {intercept:.2f}")
print(f"R-squared: {r_squared:.2f}")

# Make a prediction
hours_to_predict = [[8]] # Needs to be 2D
predicted_score = model.predict(hours_to_predict)
print(f"Predicted score for {hours_to_predict[0][0]} hours: {predicted_score[0]:.2f}")

# Plotting
plt.figure()
plt.scatter(X, y, label='Actual Scores')
plt.plot(X, model.predict(X), color='red', label='Regression Line')
plt.title("Hours Studied vs. Exam Score")
plt.xlabel("Hours Studied")
plt.ylabel("Exam Score")
plt.legend()
plt.savefig("regression_plot.png")

Learning milestones:

  1. You create a scatter plot and visually identify a trend → You can spot potential relationships in data.
  2. You can calculate and interpret the correlation coefficient → You can quantify the strength and direction of a linear relationship.
  3. You fit a linear regression model and can explain the meaning of the slope and intercept → You can model a relationship mathematically.
  4. You can use your model to make a prediction for a new data point → You have built your first predictive model.

Summary

Project Difficulty Main Language Key Learning
1. Personal Data Dashboard Beginner Python Descriptive Statistics, Visualization
2. Loot Box Simulator Beginner Python Probability, Monte Carlo Simulation
3. “Dumb” Spam Filter Intermediate Python Bayes’ Theorem, Text Classification
4. Is This Die Loaded? Intermediate Python Hypothesis Testing, Chi-Squared Test
5. Study vs. Grades Advanced Python Correlation, Linear Regression

Introduction

This additive section upgrades this guide into a complete, project-based statistics sprint with explicit coverage for all required topic clusters: mathematical foundations, probability, descriptive statistics, inference, regression, resampling, Bayesian methods, experimental design and causality, multivariate/specialized methods, practical data competence, and advanced data scientist topics.

What you will build across the expanded sprint:

  • Theory-to-practice notebooks and CLI analysis tools
  • Reproducible experiment pipelines
  • Model diagnostics reports for both frequentist and Bayesian workflows
  • A final capstone that combines experimentation, causal reasoning, and production-style reporting

Scope:

  • In scope: practical applied statistics, model assumptions, diagnostics, reproducibility, communication
  • Out of scope: measure-theoretic proofs and formal theorem-proof-heavy mathematical statistics

Big-picture architecture:

Raw Data -> Cleaning -> EDA -> Probability Model -> Inference -> Model Fit -> Diagnostics
    |          |         |            |              |            |          |
    v          v         v            v              v            v          v
 Missingness  Types   Distributions  Assumptions   Decisions    Validation  Communication

Foundation Layer (always active):
Sets/Logic + Algebra + Light Calculus + Linear Algebra + Computing Hygiene

How to Use This Guide

  • Read the Theory Primer first, but do it in passes: first skim chapter summaries, then deep-dive only the chapter needed for your next project.
  • Pick one of the recommended learning paths below instead of trying to do all projects at once.
  • For every project, finish the Thinking Exercise before implementation.
  • Use the Definition of Done checklist as a hard gate.
  • Keep one reproducibility log (dataset version, seed, assumptions, and final conclusion) for all projects.

Prerequisites & Background Knowledge

Essential Prerequisites (Must Have)

  • Basic Python or R workflow (reading CSV/parquet, loops, functions, plotting)
  • Algebra I/II comfort (equations, functions, logarithms)
  • Basic command line navigation and package management
  • Recommended reading: “Think Stats, 2nd Edition” by Allen B. Downey (Ch. 1-4)

Helpful But Not Required

  • Intro linear algebra and matrix notation
  • SQL basics for data extraction
  • Git basics for reproducible analysis history

Self-Assessment Questions

  1. Can you explain why median is more robust than mean for skewed data?
  2. Can you compute conditional probability in a simple example without looking up formulas?
  3. Can you distinguish “prediction” from “causal effect” in one sentence?

Development Environment Setup Required Tools:

  • Python 3.11+ or R 4.3+
  • JupyterLab or Quarto
  • pandas, numpy, scipy, statsmodels, scikit-learn, matplotlib, seaborn

Recommended Tools:

  • polars for faster table operations
  • duckdb for local analytical SQL
  • pytest + notebook testing (nbval) for reproducibility gates

Testing Your Setup:

$ python -m pip list | rg 'pandas|numpy|scipy|statsmodels|scikit-learn'
pandas ...
numpy ...
scipy ...
statsmodels ...
scikit-learn ...

Time Investment

  • Simple projects: 4-8 hours
  • Moderate projects: 10-20 hours
  • Complex projects: 20-40 hours
  • Full sprint: 4-6 months part-time

Important Reality Check Statistics mastery is not memorizing tests. It is disciplined thinking under uncertainty. You should expect to re-run analyses, debug assumptions, and revise conclusions as diagnostics reveal model mismatch.

Big Picture / Mental Model

Statistics as a decision engine:

Question -> Data -> Model -> Evidence -> Decision -> Monitoring
   |         |       |         |          |          |
   |         |       |         |          |          +-> drift/alerts
   |         |       |         |          +-> business policy / scientific claim
   |         |       |         +-> intervals, p-values, posterior mass
   |         |       +-> assumptions + loss function
   |         +-> measurement quality + sampling design
   +-> domain objective + error tolerance

Frequentist and Bayesian views are complementary layers:

Frequentist: "If we repeated this process, how often would this happen?"
Bayesian:    "Given the data, what should I believe now?"

Good practice: use both when possible and check if decisions agree.

Theory Primer

Concept Chapter 1: Mathematical Foundations (Set/Logic, Algebra, Light Calculus, Linear Algebra)

  • Fundamentals: Statistics is a language built on mathematical structure. Sets and logic define event spaces and valid reasoning; algebra expresses relationships and transformations; light calculus explains change and accumulation; linear algebra provides the computational backbone of modern modeling. Without these pieces, formulas look like magic symbols. With them, formulas become compressed explanations of real mechanisms. For example, sigma notation is not just shorthand; it encodes repeated accumulation that later becomes expectation and loss minimization.
  • Deep Dive: Set operations (union, intersection, complement) formalize how events combine. Logic rules guard against invalid inferential steps such as affirming the consequent. Algebraic manipulation is required when deriving estimators, confidence intervals, or closed-form regression coefficients. Exponentials and logarithms become essential once multiplicative likelihoods are transformed into additive log-likelihoods. Derivatives provide the first-order condition for optimization (critical in MLE), while integrals formalize area/probability mass and cumulative behavior. Linear algebra turns a table of observations into structured objects: vectors (single observation/feature slices) and matrices (full design matrix). Rank and invertibility determine identifiability: if columns are linearly dependent, multiple parameter vectors can explain the same signal, creating unstable estimates. Eigenvalues/eigenvectors reveal dominant directions of variance and become central in PCA and covariance analysis. This entire layer is practical: if you cannot reason about matrix conditioning, you cannot safely interpret multivariate regression.
  • How this fit on projects: directly applied in Projects 6, 7, 9, 10, 14, and 16.
  • Definitions & key terms: set, event algebra, sigma notation, invertibility, rank deficiency, eigenspace.
  • Mental model diagram: 
    Data table X (n x p)
 -> algebraic transforms (center/scale/log)
 -> optimization objective L(theta)
 -> derivative = 0 (candidate optimum)
 -> matrix solve / iterative update
 -> interpretable parameters + diagnostics
  • How it works: define objects precisely -> map to algebra -> choose objective -> check identifiability (rank/invertibility) -> solve -> verify assumptions.
  • Minimal concrete example: 
    Pseudo:
Given rows i=1..n
Compute mean mu = (1/n) * sum(x_i)
Compute centered z_i = x_i - mu
Compute variance s2 = (1/(n-1)) * sum(z_i^2)
  • Common misconceptions: “Linear algebra is only for deep learning”; “Derivatives are theoretical only”.
  • Check-your-understanding questions:
    1. Why does rank deficiency break unique coefficient estimation?
    2. Why does log transform turn products into sums?
  • Check-your-understanding answers:
    1. Non-independent columns create infinitely many equivalent solutions.
    2. log(a*b)=log(a)+log(b), enabling stable additive optimization.
  • Real-world applications: risk models, recommendation systems, sensor fusion.
  • Where you’ll apply it: Projects 6, 10, 14, 16.
  • References: Strang “Introduction to Linear Algebra”; Casella & Berger “Statistical Inference”.
  • Key insights: Mathematical structure is not optional; it is the error-correction system for statistical reasoning.
  • Summary: Foundations convert intuition into reliable inference.
  • Homework/Exercises: derive sample variance from sigma notation; explain why singular X^T X blocks OLS inverse form.
  • Solutions: variance is average squared deviation; singularity means no unique inverse due to linear dependence.

Concept Chapter 2: Probability Theory (Core Engine)

  • Fundamentals: Probability theory formalizes uncertainty. You define sample spaces, events, and probability laws, then compute how uncertainty propagates under conditions and combinations. Conditional probability and Bayes’ theorem are the two switches that power inference: conditioning updates probabilities after information arrives, while Bayes formalizes belief revision using likelihood and prior structure.
  • Deep Dive: Combinatorics supports exact counting models (permutations/combinations). Random variables map outcomes to numerical values, with PMF/PDF/CDF capturing how mass or density is distributed. Expectation is the long-run weighted average and variance quantifies dispersion around expectation. LLN explains stabilization of sample means, while CLT explains why many aggregated errors become approximately normal. Joint distributions track dependence structures, and covariance/correlation capture directional co-movement (with correlation normalized for scale). Moment generating functions provide compact access to moments and aid distributional proofs. In practice, probability is a modeling contract: once assumptions about independence/exchangeability fail, derived probabilities can be misleading.
  • How this fit on projects: Projects 7, 8, 9, 11, 12, 13.
  • Definitions & key terms: sample space, event, conditioning, prior, likelihood, posterior, joint density.
  • Mental model diagram: 
    Assumptions -> Probability Model -> Simulate/Compute -> Compare With Data -> Revise
  • How it works: define event space -> assign probability law -> compute marginals/conditionals -> test against observed frequencies.
  • Minimal concrete example: 
    Pseudo:
P(A|B) = P(A and B) / P(B)
Posterior odds = Prior odds * Likelihood ratio
  • Common misconceptions: “Correlation implies dependency in all contexts”; “Bayes ignores data”.
  • Check-your-understanding questions:
    1. Why can two variables be uncorrelated yet dependent?
    2. What does LLN guarantee and what does it not?
  • Answers:
    1. Nonlinear dependence can exist with zero linear correlation.
    2. Sample averages converge; it does not guarantee any specific finite-sample outcome.
  • Real-world applications: fraud scoring, reliability engineering, diagnostics.
  • Where you’ll apply it: Projects 7, 11, 12, 13.
  • References: Blitzstein & Hwang “Introduction to Probability”; Jaynes “Probability Theory”.
  • Key insights: Probability is the operating system beneath all statistical decisions.
  • Summary: Better assumptions -> better uncertainty quantification.
  • Homework/Exercises: derive expected number of trials for geometric event; simulate and compare.
  • Solutions: expectation is 1/p; simulations approach this as trial count grows.

Concept Chapter 3: Descriptive Statistics

  • Fundamentals: Descriptive statistics compress large datasets into interpretable summaries of location, spread, shape, and unusual observations. Mean/median/mode describe center; variance/standard deviation/IQR describe spread; skewness and kurtosis capture shape; plots reveal structure that scalar summaries miss.
  • Deep Dive: Good description is not about one “best” metric. It is about robustness and context. Mean is efficient for symmetric noise but fragile under heavy tails; median is robust but less efficient for Gaussian-like data. IQR resists outliers and supports robust anomaly screening. Histogram binning choices can create false patterns if not controlled; density plots can oversmooth and hide multimodality. Boxplots summarize quickly but can hide cluster structure. Transformations (log, Box-Cox style reasoning, z-score scaling) can stabilize variance and improve comparability, but every transform changes interpretation. Outlier detection is not automatic deletion; it is a modeling decision that must be audited. This chapter is where analysts build data skepticism.
  • How this fit on projects: Projects 8 and 15.
  • Definitions & key terms: robust estimator, skewness, kurtosis, quantile, leverage point.
  • Mental model diagram: 
    Raw column -> quality checks -> center/spread -> shape diagnostics -> transform decision
  • How it works: profile each variable -> compare robust vs non-robust summaries -> inspect plots -> document transform rationale.
  • Minimal concrete example: 
    Pseudo:
if right_skewed and strictly_positive:
  transformed = log(x)
compare sd(x) vs sd(transformed)
  • Common misconceptions: “Outlier = mistake”; “One chart is enough”.
  • Check questions: Why can two datasets share mean and variance but differ visually? Why is IQR useful with heavy tails?
  • Answers: shape/multimodality differs; IQR ignores extreme tails.
  • Real-world applications: operations dashboards, health metrics, finance monitoring.
  • Where you’ll apply it: Projects 8 and 15.
  • References: Tukey “Exploratory Data Analysis”; Wilke “Fundamentals of Data Visualization”.
  • Key insights: Description is the first model; sloppy description creates downstream inference errors.
  • Summary: EDA is model risk management, not decoration.
  • Homework: compare mean/median/IQR before and after adding two extreme points.
  • Solutions: mean shifts sharply, median/IQR move modestly.

Concept Chapter 4: Statistical Inference (Estimation, CIs, Tests, Power)

  • Fundamentals: Inference uses sample data to estimate unknown population parameters and evaluate claims under uncertainty.
  • Deep Dive: Estimation quality depends on bias-variance tradeoff, consistency, and efficiency. MLE finds parameters maximizing observed-data likelihood; method of moments equates sample moments to theoretical moments. Confidence intervals communicate precision, but interpretation matters: a 95% CI is a procedure with 95% long-run coverage, not a direct probability statement about one fixed parameter. Hypothesis testing formalizes evidence using null/alternative, test statistics, and p-values. Type I and Type II errors are operational tradeoffs tied to cost. z-tests require known or large-sample variance assumptions; t-tests handle estimated variance in means; chi-square tests cover categorical count deviations; ANOVA compares group means under shared variance structure. Power analysis should be done before data collection, not after significance hunting.
  • How this fit on projects: Projects 9 and 13.
  • Definitions & key terms: estimator, sampling distribution, CI, significance level, power, effect size.
  • Mental model diagram: 
    Sample -> Estimator -> Sampling Distribution -> Interval/Test -> Decision + Error Risk
  • How it works: formulate question -> choose estimand -> choose test/interval under assumptions -> compute -> interpret with error tradeoffs.
  • Minimal concrete example: 
    Pseudo:
if p_value < alpha:
  reject H0
else:
  fail_to_reject H0
Always report effect size + interval.
  • Common misconceptions: “p<0.05 proves the hypothesis”; “non-significant means no effect”.
  • Check questions: What changes when sample size doubles? Why can tiny effects become significant?
  • Answers: standard errors shrink; large n can detect trivial effects.
  • Real-world applications: product launches, clinical comparisons, quality control.
  • Where you’ll apply it: Projects 9, 13.
  • References: Casella & Berger; OpenIntro Statistics chapters on inference.
  • Key insights: Inference is decision theory under uncertainty, not binary truth detection.
  • Summary: Always pair significance with effect size and power context.
  • Homework: design a power plan for a two-group mean comparison.
  • Solutions: define alpha, effect size target, variance estimate, and minimum n per arm.

Concept Chapter 5: Regression & Modeling

  • Fundamentals: Regression models relationships between predictors and outcomes for explanation, prediction, or both.
  • Deep Dive: Simple and multiple linear regression estimate conditional mean structure. Assumptions (linearity, independence, homoscedasticity, normal residuals) govern inference validity. Diagnostics are non-optional: residual-vs-fitted for structure, QQ for tail behavior, leverage/Cook’s distance for influence. Multicollinearity inflates coefficient variance and destabilizes interpretations; VIF-like reasoning helps detect it. Logistic regression models log-odds for binary outcomes and should be interpreted in probabilities and odds-ratios. Regularization (Ridge/Lasso) controls variance and helps with high-dimensional settings; model selection criteria (AIC/BIC) balance fit and complexity differently.
  • How this fit on projects: Projects 10 and 16.
  • Definitions & key terms: residual, leverage, heteroscedasticity, logit, regularization path.
  • Mental model diagram: 
    Features X -> Model family -> Fit parameters -> Diagnostics -> Re-specify -> Deploy
  • How it works: define target -> fit baseline -> inspect diagnostics -> adjust features/functional form -> compare models.
  • Minimal concrete example: 
    Pseudo:
fit linear_model(y ~ x1 + x2)
if residual_pattern != random_cloud:
  add transform or interaction
  • Common misconceptions: “High R^2 means causal insight”; “regularization always improves interpretability”.
  • Check questions: Why can adding variables lower test performance? What does BIC penalize more strongly than AIC?
  • Answers: overfitting/variance; BIC imposes stronger complexity penalty as n grows.
  • Real-world applications: pricing, churn modeling, demand forecasting.
  • Where you’ll apply it: Projects 10 and 16.
  • References: ISLR (An Introduction to Statistical Learning), Gelman & Hill.
  • Key insights: Fit quality without diagnostics is untrusted output.
  • Summary: Model, diagnose, and iterate.
  • Homework: compare unregularized vs Ridge on correlated features.
  • Solutions: Ridge stabilizes coefficients and often improves out-of-sample error.

Concept Chapter 6: Resampling & Modern Methods

  • Fundamentals: Resampling replaces fragile closed-form assumptions with data-driven uncertainty estimation.
  • Deep Dive: Bootstrap approximates sampling distributions by re-sampling observations with replacement. Percentile, basic, and BCa intervals differ in bias/acceleration handling. Permutation tests build null distributions by label shuffling and are especially useful when distributional assumptions are weak. Cross-validation estimates generalization performance and helps avoid optimistic in-sample metrics. Monte Carlo simulation stress-tests decisions under uncertain inputs and scenario variation. Key invariants: resampling scheme must preserve relevant dependence structure; otherwise uncertainty estimates can be wrong.
  • How this fit on projects: Projects 11 and 16.
  • Definitions & key terms: bootstrap replicate, permutation null, fold leakage, simulation seed.
  • Mental model diagram: 
    Observed data -> resample/simulate many worlds -> metric distribution -> uncertainty-aware decision
  • How it works: define target metric -> choose resampling design -> run large repeat count -> summarize percentiles.
  • Minimal concrete example: 
    Pseudo:
for b in 1..B:
  sample_with_replacement(data)
  compute(statistic)
CI = percentile(stat_values, [2.5, 97.5])
  • Common misconceptions: “Bootstrap fixes biased data”; “CV score equals production performance”.
  • Check questions: When is permutation invalid? Why does leakage break CV?
  • Answers: invalid when exchangeability fails; leakage shares future/test information during training.
  • Real-world applications: risk bounds, robust experimentation, model comparison.
  • Where you’ll apply it: Projects 11 and 16.
  • References: Efron & Tibshirani “An Introduction to the Bootstrap”.
  • Key insights: Resampling gives empirical uncertainty, but only if the resampling design matches data structure.
  • Summary: Resampling is practical statistical insurance.
  • Homework: bootstrap median and mean on skewed data and compare interval width.
  • Solutions: median often yields wider but more robust interpretation under heavy skew.

Concept Chapter 7: Bayesian Statistics

  • Fundamentals: Bayesian analysis updates uncertainty from prior beliefs to posterior beliefs using observed data.
  • Deep Dive: The posterior is proportional to prior × likelihood. Conjugate priors produce closed forms and build intuition (e.g., Beta-Binomial). MAP and MLE differ in regularization interpretation: MAP incorporates prior information as penalty-like structure. Credible intervals express direct posterior probability statements (given model/prior). Bayesian linear regression treats coefficients as random variables and yields posterior predictive distributions that naturally encode uncertainty. MCMC (conceptual level) approximates posteriors when closed forms are unavailable; diagnostics (effective sample size, trace mixing, divergence checks) are required for trust.
  • How this fit on projects: Projects 12 and 16.
  • Definitions & key terms: prior, posterior predictive, credible interval, conjugacy, Hamiltonian Monte Carlo.
  • Mental model diagram: 
    Prior belief + Data evidence -> Posterior belief -> Decision under uncertainty
  • How it works: specify prior -> define likelihood -> compute/approximate posterior -> derive decision metric.
  • Minimal concrete example: 
    Pseudo:
prior: Beta(a,b)
observe s successes, f failures
posterior: Beta(a+s, b+f)
  • Common misconceptions: “Bayesian methods are subjective only”; “credible interval equals confidence interval”.
  • Check questions: What happens when prior is weak and sample is large? Why monitor MCMC diagnostics?
  • Answers: likelihood dominates; poor convergence invalidates posterior summaries.
  • Real-world applications: conversion optimization, adaptive trials, risk scoring.
  • Where you’ll apply it: Projects 12, 16.
  • References: McElreath “Statistical Rethinking”; Gelman et al. “Bayesian Data Analysis”.
  • Key insights: Bayesian workflows make uncertainty explicit all the way to decisions.
  • Summary: Bayesian modeling is iterative belief calibration.
  • Homework: update Beta prior after simulated campaign outcomes.
  • Solutions: add successes/failures to prior parameters and compare posterior mean shift.

Concept Chapter 8: Experimental Design & Causality

  • Fundamentals: Causal claims require design discipline, not just statistical significance.
  • Deep Dive: Randomized controlled trials provide the clearest path to causal identification by balancing confounders in expectation. Sampling strategy impacts external validity. Confounding variables create spurious associations when treatment assignment is correlated with outcome drivers. A/B testing requires randomization integrity, pre-registered metrics, and guardrails. Difference-in-differences (intro level) estimates treatment effect by comparing before-after changes across treated/control groups under parallel trends assumptions. Failures usually come from assignment leakage, metric peeking, and post-hoc subgroup mining.
  • How this fit on projects: Project 13 and Project 16.
  • Definitions & key terms: treatment effect, confounder, ignorability, parallel trends, SUTVA.
  • Mental model diagram: 
    Design choices -> identification quality -> estimable causal effect -> policy/product decision
  • How it works: define causal question -> map assumptions -> choose design (RCT/quasi-experiment) -> run diagnostics -> report sensitivity.
  • Minimal concrete example: 
    Pseudo:
DiD estimate = (Y_treated_after - Y_treated_before)
           - (Y_control_after - Y_control_before)
  • Common misconceptions: “A/B test always equals causality”; “significant uplift in one segment is enough”.
  • Check questions: Why can imbalance still happen in small randomized samples? What does parallel trends imply?
  • Answers: randomization balances in expectation, not guaranteed finite sample; untreated trends should move similarly absent treatment.
  • Real-world applications: product experiments, policy evaluation, growth analytics.
  • Where you’ll apply it: Projects 13 and 16.
  • References: Angrist & Pischke “Mostly Harmless Econometrics”; Imbens & Rubin.
  • Key insights: Design quality dominates modeling cleverness for causal reliability.
  • Summary: Causality is a design problem with statistical implementation.
  • Homework: draft an experiment plan with primary metric, power target, and guardrail metrics.
  • Solutions: must include assignment mechanism, alpha/power, and stop conditions.

Concept Chapter 9: Multivariate & Specialized Topics

  • Fundamentals: Real datasets are high-dimensional, time-dependent, and censored; specialized methods are required.
  • Deep Dive: PCA reduces dimensionality by projecting onto orthogonal directions of maximal variance. Clustering groups observations by similarity but is sensitive to scaling and distance definitions. Time-series fundamentals include trend, seasonality, autocorrelation, and non-stationarity. ARIMA (intro) models differenced autoregressive and moving-average components for forecasting under stationarity assumptions. Survival analysis handles time-to-event data with censoring; non-parametric tests provide robust alternatives when distribution assumptions fail. These methods demand diagnostic discipline: scree plots for PCA, silhouette/cluster stability for clustering, residual autocorrelation for ARIMA, and proportional hazards checks in survival settings.
  • How this fit on projects: Project 14 and Project 16.
  • Definitions & key terms: eigenvector loading, stationarity, censoring, hazard, rank-based test.
  • Mental model diagram: 
    High-dimensional / temporal data
 -> reduce / cluster / model sequence / model event-time
 -> diagnose stability
 -> deploy interpretable outputs
  • How it works: preprocess scale/time indexes -> choose method -> validate assumptions -> interpret cautiously.
  • Minimal concrete example: 
    Pseudo:
if series non_stationary:
  difference until approximately stationary
fit ARIMA(p,d,q) on transformed series
  • Common misconceptions: “PCA components are always meaningful factors”; “clusters are ground truth classes”.
  • Check questions: Why does scaling change clustering outcomes? Why differencing can help ARIMA?
  • Answers: distance metrics become dominated by large-scale features; differencing removes trend.
  • Real-world applications: customer segmentation, demand forecasts, retention/survival monitoring.
  • Where you’ll apply it: Projects 14 and 16.
  • References: Hyndman & Athanasopoulos “Forecasting: Principles and Practice”; Hastie/Tibshirani/Friedman.
  • Key insights: Specialized methods are powerful but assumption-heavy; diagnostics decide trust.
  • Summary: Method choice must follow data structure.
  • Homework: compare PCA on raw vs standardized features.
  • Solutions: standardized PCA prevents high-scale features from dominating components.

Concept Chapter 10: Practical Data Competence

  • Fundamentals: Statistical correctness fails in production if data hygiene, reproducibility, and communication are weak.
  • Deep Dive: Data cleaning includes schema validation, type correction, duplicate handling, and business-rule consistency checks. Missing data handling requires mechanism thinking (MCAR, MAR, MNAR) before choosing deletion/imputation. Feature engineering should preserve causal directionality and avoid leakage. Visualization should prioritize truthful scales, uncertainty display, and annotation of key decision thresholds. Reproducible analysis requires deterministic pipelines, versioned data, environment pinning, and auditable outputs. Communication requires translating interval-based uncertainty into action recommendations with explicit risk language.
  • How this fit on projects: Projects 15 and 16.
  • Definitions & key terms: leakage, data lineage, reproducibility contract, uncertainty communication.
  • Mental model diagram: 
    Source data -> validation -> transformation -> model -> report -> decision memo -> monitored pipeline
  • How it works: build data contract -> enforce checks -> log all transforms -> bundle outputs + assumptions.
  • Minimal concrete example: ```text Pseudo checklist:
  • set random seed
  • log dataset hash
  • save config + outputs
  • emit interpretation paragraph with caveats ```
  • Common misconceptions: “EDA notebooks are reproducible by default”; “pretty charts equal clarity”.
  • Check questions: What is target leakage? Why keep a data dictionary?
  • Answers: leakage uses future/label information at training time; dictionary preserves semantic correctness.
  • Real-world applications: analytics engineering, MLOps handoffs, audit-compliant reporting.
  • Where you’ll apply it: Projects 15 and 16.
  • References: Wickham & Grolemund “R for Data Science”; Peng “The Art of Data Science”.
  • Key insights: Reliable statistics is as much process engineering as mathematics.
  • Summary: Reproducibility and communication are first-class statistical skills.
  • Homework: build a reproducibility checklist and apply it to one prior project.
  • Solutions: include seed, version, assumptions, and deterministic rerun proof.

Concept Chapter 11: Strong Data Scientist Level (GLMs, Mixed Models, Advanced TS, Hierarchical Bayes, Learning Theory Basics)

  • Fundamentals: This layer extends core statistics to production-grade complexity and heterogeneous real-world data.
  • Deep Dive: GLMs generalize linear models through link functions and non-Gaussian distributions (Poisson for counts, Binomial for binary outcomes, Gamma for positive skewed outcomes). Mixed models handle grouped and repeated-measure data with fixed and random effects, reducing pseudo-replication and overconfident intervals. Advanced time series incorporates exogenous variables, structural breaks, state-space reasoning, and regime changes. Bayesian hierarchical models partially pool information across groups, balancing local flexibility with global stability. Statistical learning theory basics (bias-variance, capacity/complexity control, generalization bounds intuition) explain why models that fit training data can still fail operationally. Together these methods allow scalable inference where simple iid assumptions break.
  • How this fit on projects: Project 16 primarily, with support in 10, 11, 12, 14.
  • Definitions & key terms: link function, random effect, partial pooling, VC-style complexity intuition, structural break.
  • Mental model diagram: 
    Complex data regimes
 -> model family upgrade (GLM/mixed/hierarchical/advanced TS)
 -> stronger uncertainty modeling
 -> safer deployment decisions
  • How it works: classify data-generating process -> select expanded model family -> regularize/pool -> validate out-of-sample and by subgroup.
  • Minimal concrete example: 
    Pseudo:
log(E[count]) = beta0 + beta1*x + group_random_intercept
  • Common misconceptions: “Mixed models are only for academia”; “hierarchical priors always shrink too much”.
  • Check questions: Why does partial pooling reduce overfitting for sparse groups? When use Poisson vs Gaussian?
  • Answers: sparse groups borrow strength from global distribution; Poisson for count outcomes with mean-variance structure awareness.
  • Real-world applications: marketplace pricing by region, clinical center effects, demand systems.
  • Where you’ll apply it: Project 16.
  • References: Gelman & Hill; McElreath; Hyndman & Athanasopoulos.
  • Key insights: Advanced modeling is mostly about respecting data structure and uncertainty hierarchy.
  • Summary: Strong data scientist level means modeling complexity without losing interpretability.
  • Homework: design model choice matrix for binary/count/time-to-event/grouped targets.
  • Solutions: map targets to GLM/mixed/survival/time-series families and specify diagnostics.

Glossary

  • Estimand: The precise quantity you want to learn (e.g., average treatment effect, median revenue lift).
  • Exchangeability: Assumption that data points can be treated as similar for inference after conditioning.
  • Identifiability: Whether unique parameter values are recoverable from observed data.
  • Heteroscedasticity: Non-constant residual variance across fitted values.
  • Partial Pooling: Hierarchical shrinkage where group estimates borrow strength from shared structure.
  • Data Leakage: Information from the future/target contaminating training features.
  • Calibration: Agreement between predicted probabilities and observed frequencies.
  • Power: Probability of detecting an effect of a given size when it truly exists.
  • Posterior Predictive: Distribution of future outcomes under posterior uncertainty.
  • Censoring: Event time only partially observed (e.g., “not yet churned”).

Why Statistics Matters

Modern software products, public policy, medicine, and finance all rely on statistical decisions under uncertainty.

  • The U.S. Bureau of Labor Statistics projects data scientist employment to grow 36% from 2023 to 2033, much faster than average occupations (BLS OOH, accessed 2026-02-12).
  • The World Economic Forum Future of Jobs Report 2025 reports that employers expect 39% of workers’ core skills to change by 2030, increasing demand for analytical/statistical capability (WEF 2025).
  • Large-scale experimentation is a real production discipline: Microsoft reports running more than 20,000 controlled experiments per year in a major experimentation platform context (Cambridge chapter page, 2020-era reference).

Context and evolution:

  • Classic statistics started as agricultural/industrial design and quality control.
  • Modern practice adds streaming data, online experimentation, and model monitoring.
  • The core remains the same: quantify uncertainty, make decisions, and track error costs.

Old vs modern workflow:

Old (one-off report)                  Modern (continuous decision system)
Data extract -> static chart          Data contracts -> pipelines -> model + diagnostics
-> one conclusion                     -> monitored decisions -> retraining/retesting

Concept Summary Table

Concept Cluster What You Need to Internalize
Mathematical Foundations Set/logic/algebra/calculus/linear algebra are the grammar of statistical reasoning and model identifiability.
Probability Theory Events, conditional probability, Bayes, random variables, distributions, LLN/CLT power uncertainty modeling.
Descriptive Statistics Center/spread/shape summaries and visuals are hypothesis generators and quality checks.
Statistical Inference Estimation, CIs, tests, p-values, and power are decision tools with explicit error tradeoffs.
Regression & Modeling Model assumptions and diagnostics determine whether coefficients and predictions are trustworthy.
Resampling & Modern Methods Bootstrap/permutation/CV/Monte Carlo produce assumption-aware uncertainty and robustness checks.
Bayesian Statistics Priors + likelihood -> posterior supports explicit belief updating and risk-sensitive decisions.
Experimental Design & Causality Design quality determines whether your “effect” is causal or just correlated noise.
Multivariate & Specialized Topics PCA/clustering/time-series/survival/non-parametrics handle structure beyond simple iid scalar data.
Practical Data Competence Data cleaning, missingness strategy, reproducibility, and communication determine production impact.
Strong Data Scientist Layer GLMs, mixed models, advanced TS, hierarchical Bayes, and learning-theory intuition enable real-world complexity handling.

Project-to-Concept Map

Project Concepts Applied
Project 1 Descriptive Statistics, Practical Data Competence
Project 2 Probability Theory, Resampling
Project 3 Probability Theory, Bayesian Intuition, Practical Data Competence
Project 4 Statistical Inference, Probability Theory
Project 5 Regression & Modeling, Descriptive Statistics
Project 6 Mathematical Foundations
Project 7 Probability Theory
Project 8 Descriptive Statistics
Project 9 Statistical Inference
Project 10 Regression & Modeling
Project 11 Resampling & Modern Methods
Project 12 Bayesian Statistics
Project 13 Experimental Design & Causality
Project 14 Multivariate & Specialized Topics
Project 15 Practical Data Competence
Project 16 Strong Data Scientist Layer + all previous concepts

Deep Dive Reading by Concept

Concept Book and Chapter Why This Matters
Mathematical Foundations “Introduction to Linear Algebra” by Strang - Ch. 1-3 Practical matrix reasoning for regression/PCA.
Probability Theory “Introduction to Probability” by Blitzstein & Hwang - Ch. 1-8 Core event/conditional/random-variable thinking.
Descriptive Statistics “Think Stats” by Downey - Ch. 2-4 Builds practical intuition before inference.
Statistical Inference “Statistical Inference” by Casella & Berger - Ch. 7-10 Estimation and testing under uncertainty.
Regression & Modeling “An Introduction to Statistical Learning” - Ch. 3-6 Interpretable modeling and regularization.
Resampling Methods “An Introduction to the Bootstrap” by Efron & Tibshirani - Ch. 1-3 Robust uncertainty without brittle formulas.
Bayesian Statistics “Statistical Rethinking” by McElreath - Ch. 1-8 End-to-end Bayesian workflow and interpretation.
Experimental Design & Causality “Mostly Harmless Econometrics” - Ch. 2-5 Causal identification logic and pitfalls.
Multivariate & Specialized Topics “Forecasting: Principles and Practice” - TS chapters; PCA refs in ISLR Structured handling of time and high-dimensional data.
Practical Data Competence “The Art of Data Science” - entire short book Turns analysis into reproducible, communicable decisions.
Strong DS Layer “Data Analysis Using Regression and Multilevel/Hierarchical Models” Mixed/hierarchical modeling in real systems.

Quick Start: Your First 48 Hours

Day 1:

  1. Read Theory Primer chapters for Mathematical Foundations, Probability, and Descriptive Statistics.
  2. Run Project 1 or Project 6 setup and produce one validated summary table.

Day 2:

  1. Complete Project 2 or Project 7 Monte Carlo outputs and compare empirical vs theoretical probabilities.
  2. Write a one-page note explaining one model assumption you violated and how you detected it.

Path 1: The Business/Data Analyst

  • Project 1 -> Project 8 -> Project 9 -> Project 10 -> Project 11 -> Project 15

Path 2: The Product Experimenter

  • Project 2 -> Project 9 -> Project 11 -> Project 12 -> Project 13 -> Project 16

Path 3: The Data Scientist (Strong Level)

  • Project 6 -> Project 7 -> Project 10 -> Project 12 -> Project 14 -> Project 16

Success Metrics

  • You can choose and justify an inference method, not just run it.
  • You can explain uncertainty in business language with calibrated risk statements.
  • You can reproduce the same result from a fresh environment and fixed seed.
  • You can identify at least one invalid assumption in each project and show remediation.

Project Overview Table

# Project Primary Topic Difficulty Time
1 Personal Data Dashboard Descriptive Stats Beginner Weekend
2 Loot Box Simulator Probability Beginner Weekend
3 Dumb Spam Filter Conditional Probability Intermediate 1-2 weeks
4 Loaded Die Test Inference Testing Intermediate Weekend
5 Study vs Grades Regression Advanced 1-2 weeks
6 Mathematical Foundations Proving Ground Mathematical Foundations Intermediate 1 week
7 Probability Theory Engine Probability Theory Intermediate 1 week
8 Descriptive Statistics Observatory Descriptive Statistics Intermediate 1 week
9 Statistical Inference Workbench Inference Advanced 2 weeks
10 Regression & Modeling Diagnostics Lab Regression Advanced 2 weeks
11 Resampling and Modern Methods Lab Resampling Advanced 2 weeks
12 Bayesian Statistics Decision Lab Bayesian Advanced 2 weeks
13 Experimental Design and Causality Lab Causality Advanced 2 weeks
14 Multivariate & Specialized Topics Lab Multivariate/TS/Survival Expert 3 weeks
15 Practical Data Competence Pipeline Data Competence Intermediate 1-2 weeks
16 Strong Data Scientist Capstone Advanced integrated Expert 4-6 weeks

Project List

The following additive projects extend the original guide and guarantee at least one dedicated project per required topic cluster.

Project 6: Mathematical Foundations Proving Ground

  • File: LEARN_STATISTICS_FROM_SCRATCH.md
  • Main Programming Language: Python (pseudocode-first)
  • Alternative Programming Languages: R, Julia
  • Coolness Level: Level 3: Genuinely Clever
  • Business Potential: 1. The “Resume Gold”
  • Difficulty: Level 2: Intermediate
  • Knowledge Area: Mathematical Foundations
  • Software or Tool: Jupyter, SymPy (optional), NumPy
  • Main Book: “Introduction to Linear Algebra” by Gilbert Strang

What you will build: A proof-and-computation workbook that verifies core set/logic identities, function transforms, and matrix properties for statistical pipelines.

Why it teaches statistics: It removes formula memorization by making the mathematics behind estimators and models observable.

Core challenges you will face:

  • Translating set logic into event expressions -> foundations for probability rules
  • Applying logs/exponentials to likelihood functions -> foundations for MLE
  • Diagnosing rank/invertibility issues -> foundations for multivariate regression/PCA

Real World Outcome

You run a checker that prints pass/fail for identities and matrix properties you use later in modeling:

$ python foundations_workbook.py --profile stats
[PASS] DeMorgan identity checks: 200/200 random universes
[PASS] Log-likelihood transform equivalence on 5 toy datasets
[WARN] Design matrix rank deficient in scenario_03 (collinearity introduced)
[PASS] Eigen decomposition reconstructs covariance matrix within tolerance 1e-06
Report saved: outputs/foundations_report.md

The Core Question You Are Answering

“Do I actually understand the math contracts my statistical methods depend on, or am I treating formulas as magic?”

Concepts You Must Understand First

  1. Set/Logic identities and event algebra
    • Book Reference: Blitzstein & Hwang, Ch. 1
  2. Function transforms (linear, quadratic, log/exponential)
    • Book Reference: Stewart “Calculus” review chapters
  3. Matrix multiplication, rank, invertibility, eigen intuition
    • Book Reference: Strang, Ch. 1-3, 6

Questions to Guide Your Design

  1. Which identities should be tested empirically versus derived symbolically?
  2. How will you define tolerance for numeric equivalence?
  3. How will you demonstrate rank failure and its downstream effect?

Thinking Exercise

Draw a dependency graph from “event algebra” -> “conditional probability” -> “likelihood” -> “optimization” -> “regression solve” and annotate one failure mode at each edge.

The Interview Questions They Will Ask

  1. Why does rank deficiency make OLS unstable?
  2. Why are log transforms used in likelihood optimization?
  3. Give a practical example of DeMorgan’s law in event definitions.
  4. Why can eigenvectors matter in PCA?
  5. How do you detect near-singularity numerically?

Hints in Layers

Hint 1: Starting Point Use tiny 3x3 matrices and finite sample spaces first.

Hint 2: Next Level Introduce random matrix generation and automated pass/fail thresholds.

Hint 3: Technical Details Use pseudocode test harnesses to compare symbolic vs numeric outputs.

Hint 4: Tools/Debugging Track condition numbers and compare decomposition-based reconstructions.

Books That Will Help

Topic Book Chapter
Set and logic reasoning “Introduction to Probability” by Blitzstein & Hwang Ch. 1
Matrix foundations “Introduction to Linear Algebra” by Strang Ch. 1-3
Optimization intuition “Statistical Inference” by Casella & Berger Likelihood chapters

Common Pitfalls and Debugging

Problem 1: “Everything passes on tiny examples but fails on realistic matrices”

  • Why: Numerical instability hidden at small scale.
  • Fix: Add condition-number checks and tolerance-aware comparisons.
  • Quick test: Run 1,000 random matrices with logged condition numbers.

Definition of Done

  • Set/logic identity suite implemented and validated
  • Matrix rank/invertibility diagnostics demonstrated
  • Log/exp transform equivalence documented
  • Report explains at least three downstream modeling implications

Project 7: Probability Theory Engine

  • File: LEARN_STATISTICS_FROM_SCRATCH.md
  • Main Programming Language: Python (simulation-focused)
  • Alternative Programming Languages: R, Julia
  • Coolness Level: Level 4: Hardcore Tech Flex
  • Business Potential: 1. The “Resume Gold”
  • Difficulty: Level 2: Intermediate
  • Knowledge Area: Probability Theory
  • Software or Tool: NumPy, SciPy, plotting tools
  • Main Book: “Introduction to Probability” by Blitzstein & Hwang

What you will build: A reusable engine that computes and simulates sample spaces, conditional events, Bayes updates, and random variable summaries.

Why it teaches statistics: You validate analytic probability results by Monte Carlo and expose where assumptions break.

Core challenges you will face:

  • Combinatorics implementation choices -> exact vs approximate methods
  • Conditional probability handling -> event filtering correctness
  • Joint/covariance interpretation -> dependency structure understanding

Real World Outcome

$ python probability_engine.py --scenario screening_test
Scenario: disease_screening
Prior prevalence: 0.010
Sensitivity: 0.920
Specificity: 0.950
Posterior P(disease | positive): 0.157
Monte Carlo validation (2,000,000 trials): 0.158
Absolute error: 0.001
Saved: outputs/probability_engine/screening_test_report.md

The Core Question You Are Answering

“Can I reason correctly about uncertainty when priors, evidence, and dependencies all matter at once?”

Concepts You Must Understand First

  1. Sample spaces and events - Blitzstein & Hwang Ch. 1
  2. Conditional probability and Bayes’ theorem - Think Bayes Ch. 1-2
  3. Random variables, PMF/PDF/CDF, expectation/variance - OpenIntro probability chapters
  4. LLN and CLT intuition - Think Stats sampling chapters

Questions to Guide Your Design

  1. Which scenarios should be exact-counted versus simulated?
  2. How will you test independence assumptions?
  3. How will you expose confidence around Monte Carlo approximations?

Thinking Exercise

Use only pen and paper to estimate posterior probability for a low-prevalence disease test, then compare with simulation.

The Interview Questions They Will Ask

  1. Why can high test sensitivity still produce many false positives?
  2. What is the difference between correlation and dependence?
  3. How does LLN differ from CLT?
  4. Why use simulation if formulas exist?
  5. What are failure modes of naive independence assumptions?

Hints in Layers

Hint 1: Starting Point Implement one binomial scenario end-to-end first.

Hint 2: Next Level Add Bayes update module with prior and likelihood inputs.

Hint 3: Technical Details Track simulation error against theoretical values as sample size increases.

Hint 4: Tools/Debugging Use seed control and scenario snapshots for reproducibility.

Books That Will Help

Topic Book Chapter
Conditional probability “Think Bayes” by Downey Ch. 1-2
Random variables “Introduction to Probability” Ch. 3-5
LLN/CLT “Think Stats” by Downey Sampling chapters

Common Pitfalls and Debugging

Problem 1: “Simulation and theory disagree dramatically”

  • Why: Event definitions or probability normalization are wrong.
  • Fix: Add unit tests for event counts and probability sums.
  • Quick test: Verify all PMF values sum to 1.0 within tolerance.

Definition of Done

  • Engine supports conditional/Bayes scenarios
  • At least 5 scenarios validated analytically and by simulation
  • Joint distribution and covariance examples included
  • Clear report of assumptions and failure cases

Project 8: Descriptive Statistics Observatory

  • File: LEARN_STATISTICS_FROM_SCRATCH.md
  • Main Programming Language: Python
  • Alternative Programming Languages: R, SQL+BI
  • Coolness Level: Level 3: Genuinely Clever
  • Business Potential: 2. The “Micro-SaaS / Pro Tool”
  • Difficulty: Level 2: Intermediate
  • Knowledge Area: Descriptive Statistics
  • Software or Tool: pandas/polars, seaborn, matplotlib
  • Main Book: “Think Stats” by Allen B. Downey

What you will build: An EDA observatory that profiles datasets with robust and classical summaries, shape diagnostics, transformations, and outlier tags.

Why it teaches statistics: You learn why descriptive choices change downstream inference quality.

Core challenges you will face:

  • Choosing robust metrics under skew -> median/IQR vs mean/SD
  • Visual diagnostics interpretation -> histogram vs density vs boxplot
  • Transformation policy -> when log/scale helps or harms interpretation

Real World Outcome

$ python descriptive_observatory.py --input data/retail_orders.csv
Rows: 1,250,430
Columns profiled: 34
Outlier flags (IQR rule): 18,442 rows
Skewness alerts: 7 columns
Recommended transforms: 5 log, 3 standard-scale
Dashboard exported: outputs/descriptive_observatory/index.html
Summary table: outputs/descriptive_observatory/profile_summary.csv

The Core Question You Are Answering

“What does this dataset really look like before I trust any model or test built on top of it?”

Concepts You Must Understand First

  1. Mean/median/mode and robustness tradeoffs
  2. Variance, standard deviation, quantiles, IQR
  3. Skewness and kurtosis interpretation limits
  4. Visualization literacy (binning, smoothing, axis scaling)

Questions to Guide Your Design

  1. What threshold defines a “data quality warning”?
  2. How do you avoid over-flagging natural heavy tails as errors?
  3. Which outputs should be machine-readable vs human-readable?

Thinking Exercise

Take one highly skewed variable and manually compare summary interpretations before and after log transform.

The Interview Questions They Will Ask

  1. Why can mean mislead in right-skewed distributions?
  2. When is z-score outlier detection inappropriate?
  3. What does kurtosis tell you and not tell you?
  4. Why can histogram bins distort interpretation?
  5. How do you communicate outliers without overclaiming?

Hints in Layers

Hint 1: Starting Point Create one profiling table for numeric columns only.

Hint 2: Next Level Add robust alternatives and transformation suggestions.

Hint 3: Technical Details Compute before/after dispersion deltas for suggested transforms.

Hint 4: Tools/Debugging Compare outputs with and without known synthetic outliers.

Books That Will Help

Topic Book Chapter
EDA basics “Think Stats” Ch. 2-4
Visualization “Fundamentals of Data Visualization” by Wilke Selected chapters
Robust summary reasoning Tukey “Exploratory Data Analysis” Core sections

Common Pitfalls and Debugging

Problem 1: “Too many outliers, no useful signal”

  • Why: One global rule applied to heterogeneous segments.
  • Fix: Profile by cohort (region/product/time) before flagging.
  • Quick test: Compare outlier rates by segment and inspect stability.

Definition of Done

  • Profiling output includes center/spread/shape metrics
  • Robust and non-robust summaries are both reported
  • Transform recommendations include rationale
  • Outlier policy and limits are documented

Project 9: Statistical Inference Workbench

  • File: LEARN_STATISTICS_FROM_SCRATCH.md
  • Main Programming Language: Python
  • Alternative Programming Languages: R, Julia
  • Coolness Level: Level 4: Hardcore Tech Flex
  • Business Potential: 2. The “Micro-SaaS / Pro Tool”
  • Difficulty: Level 3: Advanced
  • Knowledge Area: Estimation, Confidence Intervals, Hypothesis Testing
  • Software or Tool: SciPy, statsmodels
  • Main Book: “Statistical Inference” by Casella & Berger

What you will build: A workbench that runs estimation pipelines (point, CI, bootstrap CI) and hypothesis tests (z/t/chi-square/ANOVA), with explicit power planning.

Why it teaches statistics: You practice inference as decision-making under Type I/II tradeoffs, not checkbox testing.

Core challenges you will face:

  • Estimator selection and bias-variance reasoning
  • Choosing test families from data type and assumptions
  • Power analysis before experimentation

Real World Outcome

$ python inference_workbench.py --config configs/product_uplift.yaml
Point estimate (conversion lift): 0.0132
95% CI (CLT): [0.0041, 0.0223]
95% CI (bootstrap percentile): [0.0038, 0.0227]
Two-sided z-test p-value: 0.0047
Power at observed effect and n=48000/arm: 0.91
Decision: Evidence supports uplift above practical threshold 0.5pp
Saved: outputs/inference_workbench/uplift_report.md

The Core Question You Are Answering

“Given sampling uncertainty and error costs, what conclusion is justified right now?”

Concepts You Must Understand First

  1. Point estimators, bias, variance
  2. MLE vs method of moments intuition
  3. CI construction via CLT and bootstrap
  4. Null/alternative, p-values, Type I/II errors, power

Questions to Guide Your Design

  1. What is the practical effect threshold, not just statistical threshold?
  2. Which assumptions are required for each test option?
  3. How will you prevent p-hacking and metric peeking?

Thinking Exercise

Design two conclusions for the same p-value: one with tiny practical effect and one with high practical impact. Explain decision differences.

The Interview Questions They Will Ask

  1. What does a 95% CI mean operationally?
  2. Why can a tiny effect be statistically significant?
  3. When use t-test vs z-test?
  4. Why do power analyses belong in planning, not postmortem?
  5. What does failing to reject H0 mean?

Hints in Layers

Hint 1: Starting Point Implement one pipeline for mean CI + t-test.

Hint 2: Next Level Add proportion CI and chi-square support.

Hint 3: Technical Details Add bootstrap CI and compare against CLT CI.

Hint 4: Tools/Debugging Inject synthetic known effects and verify detection rates.

Books That Will Help

Topic Book Chapter
Estimation and CI Casella & Berger Ch. 7-8
Hypothesis testing OpenIntro Statistics Inference chapters
Power analysis Cohen references + practical guides Selected sections

Common Pitfalls and Debugging

Problem 1: “Contradictory results across tests”

  • Why: Assumptions differ (variance equality, normality, sample size).
  • Fix: Add assumption diagnostics and test-selection audit trail.
  • Quick test: Run same dataset through all candidate tests with logged assumptions.

Definition of Done

  • Point and interval estimates produced for numeric and proportion metrics
  • Test selection logic documented by data conditions
  • Power module implemented and used pre-analysis
  • Decision report includes practical significance threshold

Project 10: Regression & Modeling Diagnostics Lab

  • File: LEARN_STATISTICS_FROM_SCRATCH.md
  • Main Programming Language: Python
  • Alternative Programming Languages: R, Julia
  • Coolness Level: Level 4: Hardcore Tech Flex
  • Business Potential: 2. The “Micro-SaaS / Pro Tool”
  • Difficulty: Level 3: Advanced
  • Knowledge Area: Linear/Logistic Regression, Regularization, Model Selection
  • Software or Tool: statsmodels, scikit-learn
  • Main Book: “An Introduction to Statistical Learning”

What you will build: A diagnostics lab for linear and logistic models, including assumption checks, multicollinearity reports, regularization comparison, and AIC/BIC model ranking.

Why it teaches statistics: You stop treating regression as one-fit-one-number and start treating it as an iterative model validation process.

Core challenges you will face:

  • Assumption verification pipeline design
  • Feature leakage and multicollinearity handling
  • Tradeoff between interpretability and performance

Real World Outcome

$ python regression_diagnostics_lab.py --dataset data/customer_churn.parquet
Model family: logistic
AIC(best): 12431.8  BIC(best): 12607.4
Top risk features: tenure_months (-), support_tickets (+), contract_type
VIF alert: billing_cycle and annual_plan_flag (VIF > 9)
Calibration slope: 0.96
Saved residual and calibration diagnostics to outputs/regression_lab/
Decision memo: outputs/regression_lab/model_readout.md

The Core Question You Are Answering

“Which model is trustworthy enough to guide action, given assumptions, diagnostics, and business constraints?”

Concepts You Must Understand First

  1. Linear and logistic regression mechanics
  2. Assumptions: linearity, independence, homoscedasticity, normality (for linear inference)
  3. Multicollinearity and variance inflation
  4. Regularization (Ridge/Lasso) and AIC/BIC

Questions to Guide Your Design

  1. Which diagnostic failures are blocking vs advisory?
  2. How will you compare regularized and unregularized models fairly?
  3. How will you present uncertainty and calibration to stakeholders?

Thinking Exercise

Given two models where one has better AUC but worse calibration, decide which to deploy for a retention campaign and defend your choice.

The Interview Questions They Will Ask

  1. What does multicollinearity break and what does it not break?
  2. When is AIC preferred over BIC?
  3. Why inspect residual plots even with high R-squared?
  4. How is logistic coefficient interpretation different from linear slope?
  5. Why might regularization improve out-of-sample performance?

Hints in Layers

Hint 1: Starting Point Build baseline linear/logistic fit with train/validation split.

Hint 2: Next Level Add diagnostics and collinearity checks.

Hint 3: Technical Details Integrate Ridge/Lasso path and compare AIC/BIC style summaries.

Hint 4: Tools/Debugging Use synthetic data where true coefficients are known to validate interpretation.

Books That Will Help

Topic Book Chapter
Linear regression ISLR Ch. 3
Classification/logistic ISLR Ch. 4
Multilevel regression intuition Gelman & Hill Early chapters

Common Pitfalls and Debugging

Problem 1: “Great validation metric but unstable coefficients”

  • Why: Collinearity or leakage.
  • Fix: Remove redundant predictors, enforce leakage audits, compare coefficient stability across folds.
  • Quick test: Refit across bootstrap samples and inspect coefficient variance.

Definition of Done

  • Linear and logistic pipelines implemented
  • Diagnostic artifacts generated and interpreted
  • Multicollinearity and regularization analysis completed
  • Final model selection justified with AIC/BIC + operational constraints

Project 11: Resampling and Modern Methods Lab

  • File: LEARN_STATISTICS_FROM_SCRATCH.md
  • Main Programming Language: Python
  • Alternative Programming Languages: R
  • Coolness Level: Level 4: Hardcore Tech Flex
  • Business Potential: 1. The “Resume Gold”
  • Difficulty: Level 3: Advanced
  • Knowledge Area: Bootstrap, Permutation, Cross-Validation, Monte Carlo
  • Software or Tool: NumPy, scikit-learn
  • Main Book: “An Introduction to the Bootstrap” by Efron & Tibshirani

What you will build: A resampling lab that quantifies uncertainty for metrics and model comparisons under minimal distribution assumptions.

Why it teaches statistics: It makes uncertainty tangible through repeated synthetic worlds rather than one-shot estimates.

Core challenges you will face:

  • Choosing valid resampling strategies for each data structure
  • Avoiding data leakage in CV workflows
  • Interpreting interval overlap and practical equivalence

Real World Outcome

$ python resampling_lab.py --task model_compare
Bootstrap replicates: 5000
Permutation replicates: 10000
Model A AUC 95% CI: [0.782, 0.811]
Model B AUC 95% CI: [0.776, 0.807]
Permutation p-value (A better than B): 0.118
Recommendation: models statistically indistinguishable at alpha=0.05
Saved: outputs/resampling_lab/comparison_report.md

The Core Question You Are Answering

“How confident should I be that this observed difference is real and not sampling luck?”

Concepts You Must Understand First

  1. Bootstrap intervals and bias notions
  2. Permutation null distributions
  3. Cross-validation fold design and leakage
  4. Monte Carlo scenario simulation

Questions to Guide Your Design

  1. How many replicates are enough for stable conclusions?
  2. What assumptions does each resampling method require?
  3. Which metric should be resampled (mean, median, AUC, uplift)?

Thinking Exercise

Design one scenario where bootstrap is appropriate and one where naive bootstrap is invalid due to dependence.

The Interview Questions They Will Ask

  1. Why might bootstrap and parametric CI disagree?
  2. What is exchangeability in permutation tests?
  3. How can CV leakage happen in feature engineering?
  4. Why does random seed governance matter?
  5. What is Monte Carlo useful for in product decisions?

Hints in Layers

Hint 1: Starting Point Bootstrap one scalar metric first.

Hint 2: Next Level Add permutation significance for model deltas.

Hint 3: Technical Details Implement repeated CV with controlled split registry.

Hint 4: Tools/Debugging Track convergence of interval endpoints as replicate count grows.

Books That Will Help

Topic Book Chapter
Bootstrap Efron & Tibshirani Ch. 1-3
CV/model assessment ISLR Resampling chapter
Simulation thinking Think Stats Simulation chapters

Common Pitfalls and Debugging

Problem 1: “Intervals change wildly between runs”

  • Why: Too few replicates or uncontrolled randomness.
  • Fix: Increase replicates and enforce deterministic seeds.
  • Quick test: Run 5 repeats and verify interval endpoint drift is within tolerance.

Definition of Done

  • Bootstrap and permutation modules implemented
  • Repeated CV with leakage checks in place
  • Monte Carlo scenario report generated
  • Final recommendation includes uncertainty interpretation

Project 12: Bayesian Statistics Decision Lab

  • File: LEARN_STATISTICS_FROM_SCRATCH.md
  • Main Programming Language: Python
  • Alternative Programming Languages: R, Stan-based workflow
  • Coolness Level: Level 4: Hardcore Tech Flex
  • Business Potential: 2. The “Micro-SaaS / Pro Tool”
  • Difficulty: Level 3: Advanced
  • Knowledge Area: Priors, Posterior, Credible Intervals, Bayesian Regression
  • Software or Tool: PyMC/Stan (conceptual usage)
  • Main Book: “Statistical Rethinking” by Richard McElreath

What you will build: A Bayesian decision lab for conversion-rate and revenue-impact inference with prior sensitivity checks and posterior predictive diagnostics.

Why it teaches statistics: You learn explicit uncertainty language for decision thresholds and risk tolerance.

Core challenges you will face:

  • Prior selection and transparency
  • MAP vs MLE interpretation differences
  • MCMC convergence diagnostics (conceptual level)

Real World Outcome

$ python bayesian_decision_lab.py --campaign spring_launch
Posterior P(variant_B > variant_A): 0.974
95% credible interval for uplift: [0.003, 0.021]
Expected monthly incremental conversions: 412
Prior sensitivity: low (decision stable across 3 prior families)
Posterior predictive check: pass
Saved: outputs/bayesian_lab/decision_brief.md

The Core Question You Are Answering

“Given what we believed before and what we observed now, what should we believe and do next?”

Concepts You Must Understand First

  1. Prior-likelihood-posterior workflow
  2. Conjugate prior intuition (Beta-Binomial)
  3. MAP vs MLE
  4. Credible intervals and posterior predictive checks

Questions to Guide Your Design

  1. Which prior is defensible for this domain?
  2. How do you detect posterior instability?
  3. What probability threshold triggers action?

Thinking Exercise

Create two priors (skeptical and optimistic) and reason how much data is needed for both to converge toward the same decision.

The Interview Questions They Will Ask

  1. What is the practical difference between credible and confidence intervals?
  2. Why do priors not “override” large datasets in many cases?
  3. What indicates MCMC convergence problems?
  4. Why use posterior predictive checks?
  5. When might MAP be preferable to MLE?

Hints in Layers

Hint 1: Starting Point Implement Beta-Binomial update for a single binary metric.

Hint 2: Next Level Add posterior decision thresholds and expected utility outputs.

Hint 3: Technical Details Introduce MCMC for non-conjugate variants and inspect trace diagnostics.

Hint 4: Tools/Debugging Run prior sensitivity sweeps and compare decision stability.

Books That Will Help

Topic Book Chapter
Bayesian fundamentals “Statistical Rethinking” Ch. 1-5
Hierarchical extensions “Bayesian Data Analysis” Hierarchical chapters
Applied Bayesian workflow domain-specific notebooks selected references

Common Pitfalls and Debugging

Problem 1: “Posterior seems too certain”

  • Why: Misspecified likelihood or poor priors.
  • Fix: Run posterior predictive diagnostics and prior predictive checks.
  • Quick test: Simulate from prior and posterior predictive distributions and compare to observed scale.

Definition of Done

  • Prior-likelihood-posterior pipeline implemented
  • Credible intervals and decision probabilities reported
  • Prior sensitivity analysis included
  • MCMC diagnostics documented (if non-conjugate models used)

Project 13: Experimental Design and Causality Lab

  • File: LEARN_STATISTICS_FROM_SCRATCH.md
  • Main Programming Language: Python + SQL
  • Alternative Programming Languages: R
  • Coolness Level: Level 5: Pure Magic
  • Business Potential: 3. Service & Support
  • Difficulty: Level 3: Advanced
  • Knowledge Area: RCTs, Sampling, Confounding, A/B Testing, Difference-in-Differences
  • Software or Tool: SQL warehouse, stats toolkit
  • Main Book: “Mostly Harmless Econometrics”

What you will build: A causality lab that compares randomized A/B outcomes with quasi-experimental DiD estimates on synthetic policy/product interventions.

Why it teaches statistics: It teaches when causal claims are justified and how to communicate uncertainty in causal language.

Core challenges you will face:

  • Randomization integrity checks
  • Confounder diagnostics for observational settings
  • Parallel trends assessment in DiD

Real World Outcome

$ python causality_lab.py --case pricing_rollout
RCT estimated treatment effect: +1.8pp conversion (95% CI: +0.9pp, +2.7pp)
Observational naive estimate: +3.4pp
DiD estimate: +1.6pp (parallel trends check: acceptable)
Conclusion: naive observational estimate confounded upward
Saved causal brief: outputs/causality_lab/pricing_rollout_brief.md

The Core Question You Are Answering

“Is this observed uplift actually caused by the intervention, or is it confounding dressed as impact?”

Concepts You Must Understand First

  1. Randomized controlled trial logic
  2. Sampling methods and external validity
  3. Confounding and selection bias
  4. Difference-in-differences assumptions

Questions to Guide Your Design

  1. What randomization unit is appropriate: user, session, cluster?
  2. Which guardrail metrics can detect harmful side effects?
  3. How will you check pre-trends for DiD validity?

Thinking Exercise

Take one historical product change and write both a non-causal and causal interpretation. Explain what design evidence would separate them.

The Interview Questions They Will Ask

  1. Why is randomization powerful but not infallible?
  2. What does parallel trends mean in DiD?
  3. How do you detect sample ratio mismatch in A/B tests?
  4. What is a confounder in practical product analytics?
  5. How do you choose primary and guardrail metrics?

Hints in Layers

Hint 1: Starting Point Build assignment and balance diagnostics first.

Hint 2: Next Level Add a standard A/B estimator with interval output.

Hint 3: Technical Details Implement a DiD module and include pre-trend diagnostics.

Hint 4: Tools/Debugging Simulate known confounding to verify your detection logic.

Books That Will Help

Topic Book Chapter
Causal inference basics “Mostly Harmless Econometrics” Ch. 2-5
Experimental design “Trustworthy Online Controlled Experiments” Selected chapters
Applied A/B testing experimentation platform docs selected sections

Common Pitfalls and Debugging

Problem 1: “A/B test says significant, but rollout fails”

  • Why: Sample mismatch, novelty effects, or metric leakage.
  • Fix: Add assignment integrity checks and post-period holdout validation.
  • Quick test: Verify treatment/control allocation and pre-period metric parity.

Definition of Done

  • RCT pipeline with assignment diagnostics implemented
  • Confounding-aware observational comparison included
  • DiD module with pre-trend checks completed
  • Causal decision brief with assumptions and caveats delivered

Project 14: Multivariate & Specialized Topics Lab

  • File: LEARN_STATISTICS_FROM_SCRATCH.md
  • Main Programming Language: Python
  • Alternative Programming Languages: R
  • Coolness Level: Level 5: Pure Magic
  • Business Potential: 2. The “Micro-SaaS / Pro Tool”
  • Difficulty: Level 4: Expert
  • Knowledge Area: PCA, Clustering, Time Series, ARIMA, Survival, Non-parametrics
  • Software or Tool: scikit-learn, statsmodels, lifelines
  • Main Book: ISLR + Forecasting: Principles and Practice

What you will build: A multi-method lab that runs dimensionality reduction, segmentation, forecasting, and time-to-event analyses on one cohesive business dataset.

Why it teaches statistics: You learn to match method choice to data structure rather than forcing one model family.

Core challenges you will face:

  • Scaling and distance sensitivity in PCA/clustering
  • Stationarity and residual diagnostics in ARIMA
  • Censoring-aware interpretation in survival analysis

Real World Outcome

$ python multivariate_lab.py --dataset data/subscription_platform.parquet
PCA retained components: 6 (explained variance: 82.4%)
Best cluster count (silhouette): 4
ARIMA forecast MAPE (next 8 weeks): 6.8%
Median survival time by segment: segment_2 = 11.4 months
Non-parametric test (Kruskal-Wallis) p-value: 0.012
Saved full report: outputs/multivariate_lab/analysis_report.md

The Core Question You Are Answering

“Which specialized method best fits each data structure in this problem, and how do I verify it?”

Concepts You Must Understand First

  1. Eigenvectors/eigenvalues and PCA loadings
  2. Distance metrics and clustering stability
  3. Time-series components and ARIMA basics
  4. Censoring and survival curve interpretation
  5. Non-parametric alternatives to parametric tests

Questions to Guide Your Design

  1. How will you standardize features before PCA/clustering?
  2. What diagnostics define an acceptable forecast model?
  3. When should non-parametric tests replace ANOVA/t-tests?

Thinking Exercise

Build a “method selection table” from target type and data properties (static, temporal, censored, high-dimensional).

The Interview Questions They Will Ask

  1. Why can PCA hurt interpretability?
  2. What does stationarity mean for ARIMA?
  3. Why are cluster labels not ground truth classes?
  4. What is right-censoring in survival data?
  5. When use Kruskal-Wallis instead of ANOVA?

Hints in Layers

Hint 1: Starting Point Separate the project into four tracks: PCA/cluster/forecast/survival.

Hint 2: Next Level Create standardized diagnostics per track.

Hint 3: Technical Details Use holdout windows for forecasting and cluster stability checks across seeds.

Hint 4: Tools/Debugging Stress-test with synthetic datasets where true structure is known.

Books That Will Help

Topic Book Chapter
PCA and clustering ISLR Unsupervised learning chapter
Time series and ARIMA Hyndman & Athanasopoulos ARIMA chapters
Survival basics applied biostatistics references survival chapters

Common Pitfalls and Debugging

Problem 1: “Cluster story changes every run”

  • Why: Unstable initialization and weak cluster structure.
  • Fix: Use multiple restarts and report stability metrics.
  • Quick test: Repeat clustering over 50 seeds and compute assignment consistency.

Definition of Done

  • PCA + clustering + ARIMA + survival components implemented
  • Diagnostics for each specialized method included
  • Non-parametric test path added for assumption failures
  • Integrated interpretation memo produced

Project 15: Practical Data Competence Pipeline

  • File: LEARN_STATISTICS_FROM_SCRATCH.md
  • Main Programming Language: Python + SQL
  • Alternative Programming Languages: R, dbt-style SQL workflows
  • Coolness Level: Level 3: Genuinely Clever
  • Business Potential: 3. Service & Support
  • Difficulty: Level 2: Intermediate
  • Knowledge Area: Data Cleaning, Missing Data, Feature Engineering, Visualization, Reproducibility, Communication
  • Software or Tool: DuckDB/SQL, notebook pipeline, reporting templates
  • Main Book: “The Art of Data Science”

What you will build: A production-style analysis pipeline that ingests messy data, enforces quality checks, handles missingness, engineers features, and emits a reproducible executive memo.

Why it teaches statistics: It operationalizes statistical work so results remain trustworthy outside your local notebook.

Core challenges you will face:

  • Designing a missing-data strategy aligned with mechanism assumptions
  • Avoiding leakage during feature engineering
  • Writing concise, uncertainty-aware stakeholder communication

Real World Outcome

$ python practical_pipeline.py --run weekly_kpi
Input rows: 4,822,194
Schema violations fixed: 1,402
Missingness strategy: MAR-impute on 6 fields, drop on 2 fields
Feature checks: 18 passed, 1 warning (potential leakage)
Reproducibility hash: 6fa2a9d1
Executive memo generated: outputs/practical_pipeline/weekly_kpi_brief.md
Artifact bundle: outputs/practical_pipeline/run_2026_02_12.tar.gz

The Core Question You Are Answering

“Can another analyst rerun my workflow next month and reach the same conclusion with an auditable trail?”

Concepts You Must Understand First

  1. Data quality dimensions and validation gates
  2. Missingness mechanisms (MCAR, MAR, MNAR)
  3. Feature leakage and temporal cutoff discipline
  4. Visualization best practices and uncertainty communication

Questions to Guide Your Design

  1. Which quality failures should block the pipeline?
  2. How will you justify imputation choices?
  3. What format makes your results decision-ready for non-technical stakeholders?

Thinking Exercise

Write two one-paragraph summaries of the same analysis: one for data scientists and one for executives. Compare what changed.

The Interview Questions They Will Ask

  1. How do you detect target leakage?
  2. What is your process for missing data decisions?
  3. What makes an analysis reproducible?
  4. How do you communicate uncertainty without confusing stakeholders?
  5. What does a data contract include?

Hints in Layers

Hint 1: Starting Point Create deterministic input validation and logging first.

Hint 2: Next Level Add missingness report and feature engineering audit.

Hint 3: Technical Details Package outputs with run metadata hash and assumptions manifest.

Hint 4: Tools/Debugging Re-run in a clean environment and verify hash and outputs match.

Books That Will Help

Topic Book Chapter
Data workflow rigor “The Art of Data Science” Full book
Reproducible pipelines “R for Data Science” Workflow chapters
Communication data storytelling references selected sections

Common Pitfalls and Debugging

Problem 1: “Notebook works once, fails on rerun”

  • Why: Hidden state and implicit execution order.
  • Fix: Convert notebook logic into ordered pipeline steps with explicit dependencies.
  • Quick test: Run from clean environment twice and compare artifact hashes.

Definition of Done

  • Validation, missingness, and feature checks implemented
  • Reproducibility manifest and run hash produced
  • Visualizations include uncertainty-aware annotations
  • Executive memo includes assumptions, limits, and recommendation

Project 16: Strong Data Scientist Capstone

  • File: LEARN_STATISTICS_FROM_SCRATCH.md
  • Main Programming Language: Python + SQL
  • Alternative Programming Languages: R/Stan ecosystem
  • Coolness Level: Level 5: Pure Magic
  • Business Potential: 2. The “Micro-SaaS / Pro Tool”
  • Difficulty: Level 4: Expert
  • Knowledge Area: GLMs, Mixed Models, Advanced Time Series, Bayesian Hierarchical Models, Statistical Learning Theory Basics
  • Software or Tool: statsmodels, PyMC/Stan, forecasting library, experiment platform logs
  • Main Book: Gelman & Hill + Statistical Rethinking

What you will build: A unified decision system for a multi-region subscription product combining causal experimentation, mixed-effects conversion modeling, hierarchical Bayesian forecasting, and uncertainty-aware policy simulation.

Why it teaches statistics: It integrates everything into one realistic data scientist-level workflow where model choice, design assumptions, and communication all matter.

Core challenges you will face:

  • Choosing GLM/mixed/hierarchical models by data structure
  • Reconciling frequentist and Bayesian conclusions for policy decisions
  • Balancing model complexity with generalization and interpretability

Real World Outcome

$ python ds_capstone.py --scenario global_pricing
GLM (Poisson) demand elasticity estimated for 12 regions
Mixed model random-effect variance (region): 0.43
Hierarchical Bayesian posterior P(policy_improves_margin): 0.948
Advanced TS forecast interval coverage (8-week): 93.1%
Policy simulator expected annual margin delta: +$2.8M (P10: +$0.9M, P90: +$4.4M)
Final decision packet: outputs/ds_capstone/final_policy_packet.md

The Core Question You Are Answering

“Can I deliver a production-grade statistical recommendation that remains robust across model families, segments, and future uncertainty?”

Concepts You Must Understand First

  1. GLM families and link functions
  2. Mixed-effects modeling and partial pooling
  3. Advanced time-series diagnostics and structural change
  4. Bayesian hierarchical modeling and posterior predictive checks
  5. Bias-variance/generalization tradeoffs from statistical learning theory basics

Questions to Guide Your Design

  1. Which model family matches each target and data hierarchy?
  2. How will you validate subgroup fairness and calibration?
  3. What uncertainty format will decision-makers trust?

Thinking Exercise

Create a “model governance one-pager” listing assumptions, diagnostics, and failure triggers for GLM, mixed, TS, and Bayesian components.

The Interview Questions They Will Ask

  1. Why use mixed models instead of fixed-effects-only regression?
  2. What is partial pooling and why is it valuable for sparse groups?
  3. How do you detect structural breaks in time-series forecasting?
  4. How do you compare Bayesian and frequentist evidence in one report?
  5. What generalization risks remain after cross-validation?

Hints in Layers

Hint 1: Starting Point Ship a narrow vertical slice: one region, one target, one uncertainty report.

Hint 2: Next Level Add hierarchical structure and region-level partial pooling.

Hint 3: Technical Details Introduce Bayesian posterior predictive checks and advanced forecast residual diagnostics.

Hint 4: Tools/Debugging Run ablation studies to see which modeling layer changes the decision most.

Books That Will Help

Topic Book Chapter
GLMs and mixed models “Data Analysis Using Regression and Multilevel/Hierarchical Models” Core chapters
Bayesian hierarchy “Statistical Rethinking” Hierarchical chapters
Advanced forecasting “Forecasting: Principles and Practice” Advanced sections

Common Pitfalls and Debugging

Problem 1: “Complex model improves fit but hurts operational trust”

  • Why: Interpretability and calibration checks were under-prioritized.
  • Fix: Add model cards, subgroup diagnostics, and simpler benchmark comparisons.
  • Quick test: Compare policy recommendation stability across simplified baselines.

Definition of Done

  • GLM + mixed + Bayesian hierarchical + advanced TS modules integrated
  • Cross-model decision consistency analysis completed
  • Uncertainty-aware policy simulation delivered
  • Final packet includes assumptions, diagnostics, and risk-governed recommendation

Project Comparison Table

Project Difficulty Time Depth of Understanding Fun Factor
6. Mathematical Foundations Proving Ground Intermediate 1 week High mathematical clarity ★★★★☆
7. Probability Theory Engine Intermediate 1 week Core uncertainty reasoning ★★★★☆
8. Descriptive Statistics Observatory Intermediate 1 week Strong EDA instincts ★★★☆☆
9. Statistical Inference Workbench Advanced 2 weeks Decision-quality inference ★★★★★
10. Regression & Modeling Diagnostics Lab Advanced 2 weeks Model trust and diagnostics ★★★★★
11. Resampling and Modern Methods Lab Advanced 2 weeks Robust uncertainty estimation ★★★★☆
12. Bayesian Statistics Decision Lab Advanced 2 weeks Posterior decision thinking ★★★★★
13. Experimental Design and Causality Lab Advanced 2 weeks Causal reliability ★★★★★
14. Multivariate & Specialized Topics Lab Expert 3 weeks Method-selection maturity ★★★★★
15. Practical Data Competence Pipeline Intermediate 1-2 weeks Production analytics discipline ★★★★☆
16. Strong Data Scientist Capstone Expert 4-6 weeks End-to-end data scientist mastery ★★★★★

Recommendation

If you are new to statistics: start with Project 6, then Project 8, then Project 9.

If you are already an analyst: start with Project 9 and Project 15 to strengthen inference quality and reproducibility.

If you want strong data scientist level: follow Project 10 -> 11 -> 12 -> 13 -> 14 -> 16.

Final Overall Project: Statistical Decision Engine for AI Products

The Goal: combine Projects 9, 11, 12, 13, 15, and 16 into one decision engine for product launches.

  1. Build data contracts and reproducible ingestion.
  2. Run descriptive + inference + Bayesian views in parallel.
  3. Simulate rollout policy risk using resampling and scenario forecasts.
  4. Publish one decision memo with explicit uncertainty and guardrails.

Success Criteria: the same pipeline reruns deterministically, model diagnostics pass quality gates, and final recommendation includes quantified downside risk.

From Learning to Production

Your Project Production Equivalent Gap to Fill
Project 9 Inference Workbench Experiment analysis service automated metric catalog + governance
Project 10 Regression Lab Production scoring service feature store + model monitoring
Project 12 Bayesian Lab Adaptive decision engine prior governance + posterior serving
Project 13 Causality Lab Experimentation platform analytics assignment integrity automation
Project 15 Practical Pipeline Analytics engineering stack orchestration, CI tests, access controls
Project 16 Capstone Decision intelligence platform scalability, reliability SLOs, auditability

Summary

This additive expansion now covers the full statistics path from foundations to strong data scientist-level practice, while preserving all original content.

# Project Name Main Language Difficulty Time Estimate
6 Mathematical Foundations Proving Ground Python Intermediate 1 week
7 Probability Theory Engine Python Intermediate 1 week
8 Descriptive Statistics Observatory Python Intermediate 1 week
9 Statistical Inference Workbench Python Advanced 2 weeks
10 Regression & Modeling Diagnostics Lab Python Advanced 2 weeks
11 Resampling and Modern Methods Lab Python Advanced 2 weeks
12 Bayesian Statistics Decision Lab Python Advanced 2 weeks
13 Experimental Design and Causality Lab Python/SQL Advanced 2 weeks
14 Multivariate & Specialized Topics Lab Python Expert 3 weeks
15 Practical Data Competence Pipeline Python/SQL Intermediate 1-2 weeks
16 Strong Data Scientist Capstone Python/SQL Expert 4-6 weeks

Expected Outcomes

  • You can choose and justify statistical methods by assumptions and data structure.
  • You can design trustworthy experiments and causal analyses.
  • You can ship reproducible, uncertainty-aware recommendations.

Additional Resources and References

Standards and Specifications

Industry Analysis

Books

  • “Think Stats” by Allen B. Downey - practical statistical intuition through computation
  • “Statistical Inference” by Casella & Berger - rigorous inference foundations
  • “An Introduction to Statistical Learning” - applied regression/modeling diagnostics
  • “Statistical Rethinking” by Richard McElreath - Bayesian modeling and decision framing