Module 9 — Expectation, variance and common distributions

A random variable — the outcome of a draw, the value of a feature — is summarized by two numbers: where it sits on average (expectation) and how much it scatters (variance). This module gives these two summaries and presents the few probability distributions that recur constantly in data science.

Expectation: the center of gravity

The expectation $E[X]$ is the average value of a random variable over many draws — the center of gravity of its distribution. For a fair die it equals 3.5: not a possible face, but the long-run average.

$$E[X] = \sum_i x_i \, P(x_i)$$

On data, expectation is estimated simply by the empirical mean (df["column"].mean()). It is the first number you look at, but it never suffices: two datasets can share the same mean and have nothing in common.

Variance and standard deviation: the spread

Variance measures the scatter around the expectation: the mean of the squared deviations from the center.

$$\text{Var}(X) = E[(X - E[X])^2]$$

Because it is expressed in a squared unit (euros², meters²), we often prefer the standard deviation $\sigma$, its square root, back in the original unit and thus directly interpretable.

import numpy as np
grades = np.array([10, 12, 11, 9, 13])
grades.mean()   # estimated expectation: 11.0
grades.std()    # standard deviation: spread around 11

The canonical example: two classes both average 11, but one ranges from 10 to 12 (low variance, homogeneous group) and the other from 2 to 20 (high variance, scattered group). Expectation confuses them; variance tells them apart. This center/spread duality pervades the next module, on the bias-variance trade-off.

Three distributions to recognize

A probability distribution describes how the values of a random variable are spread. Three recur constantly:

Distribution	Models	Example
Bernoulli	a binary event (success/failure)	click or not, spam or not
Uniform	all values equally likely	a die, a random draw
Normal (Gaussian)	values clustered around a mean	heights, measurement errors

The normal distribution: the queen of distributions

The normal distribution, a symmetric bell curve, describes a host of natural phenomena and holds a central place in statistics. Its shape is fully fixed by two parameters: the mean $\mu$ (where the peak is) and the standard deviation $\sigma$ (its width). A handy rule, the "68-95-99.7 rule," is worth memorizing:

• 68% of values fall within one standard deviation of the mean.
• 95% within two standard deviations.
• 99.7% within three.

This rule turns the standard deviation into a detection tool: a value more than three standard deviations from the mean is so improbable (less than 0.3%) that it is readily treated as an anomaly — a common outlier-detection method, echoing the data cleaning in the Python course.

Why the normal shows up everywhere

The central limit theorem explains this ubiquity: the mean of a large number of independent draws tends toward a normal distribution, whatever the starting distribution. That's why measurement errors, sample means and countless aggregated quantities follow a bell curve. It's also the foundation of the confidence intervals and hypothesis tests used to validate an A/B test.

Summary

• Expectation is a random variable's center of gravity; on data it's the empirical mean — necessary but never sufficient.
• Variance measures spread; the standard deviation, its root, is interpretable in the original unit.
• Three key distributions: Bernoulli (binary), uniform (equally likely), normal (bell-shaped).
• The normal is fixed by its mean and standard deviation; the 68-95-99.7 rule makes it an anomaly-detection tool, and the central limit theorem explains its ubiquity.

Next module: the bias-variance trade-off, where these notions of spread finally illuminate why a model over- or under-fits.