Lesson 4 — Probability and statistics
Linear algebra gives you the language and calculus gives you training. Probability and statistics give you judgement, and they are the branches practitioners most often skip. The result is confident conclusions drawn from noise, which is a worse failure than a badly tuned model because nobody notices.
A prediction is a distribution, not a verdict
The first mental shift: a classifier does not output a class. It outputs a probability for each class, and someone then applies a threshold to get a decision.
A model shown a photograph does not say "dog". It says something closer to: dog 0.82, wolf 0.11, fox 0.05, cat 0.02. The label "dog" appears only because a piece of code picked the highest value.
This matters because the distribution carries information the label throws away.
- 0.82 dog, 0.11 wolf: the model is fairly confident
- 0.35 dog, 0.34 wolf, 0.31 fox: the model has no idea, and reporting "dog" is close to dishonest
Systems that ignore this treat a coin flip as an answer. Systems that use it can escalate uncertain cases to a human, which is usually the single highest-value design decision in an applied AI product.
It means that among all the cases where the model says 0.7, roughly 70% should turn out positive. If in reality only 40% do, the model is poorly calibrated: its numbers are rankings dressed up as probabilities. Calibration is checkable, frequently bad, and routinely ignored.
The threshold is a business decision
Since the model gives a probability and the decision needs a cut-off, someone must choose where to cut. That choice is not statistical; it is about which mistake costs more.
Two kinds of error, and they trade off against each other:
- False positive: the model says yes, reality says no. A healthy patient told they may be ill.
- False negative: the model says no, reality says yes. A sick patient told they are fine.
Lower the threshold and you catch more real cases while raising false alarms. Raise it and you reduce false alarms while missing more real cases. There is no setting that reduces both, and pretending otherwise is where a lot of bad product decisions come from.
Which is why the vocabulary exists:
| Metric | What it answers | Matters most when |
|---|---|---|
| Accuracy | what fraction did it get right overall | classes are balanced and errors cost the same |
| Precision | when it says yes, how often is it right | false alarms are expensive: spam filtering |
| Recall | of all real cases, how many did it catch | misses are expensive: disease screening |
| F1 | a single blend of precision and recall | you need one number and both matter |
| ROC AUC | how well it ranks, across all thresholds | comparing models independently of threshold |
Why accuracy misleads
The illustration everyone should see once. Suppose a disease affects 1 in 1000 people, and you build a model that always says "healthy".
Its accuracy is 99.9%. It has never detected a single case. It is worthless.
On imbalanced data — fraud, disease, defects, churn, essentially every interesting problem — accuracy is close to meaningless, because the majority class dominates it. Precision and recall are what you look at, and quoting accuracy on an imbalanced problem is a reliable sign that someone has not thought about it.
Bayes, in plain language
Bayes' theorem sounds intimidating and says something simple: update what you believed in light of new evidence, in proportion to how strong the evidence is.
The worked example that fixes it permanently. A test for a disease is 99% accurate. You test positive. What is the chance you are ill?
The intuitive answer is 99%. The correct answer, if the disease affects 1 in 10,000 people, is roughly 1%.
Why: out of a million people, about 100 are genuinely ill and the test catches about 99 of them. The other 999,900 are healthy, and 1% of them — about 9,999 people — test positive anyway. So among roughly 10,098 positive results, only 99 are real. Your positive result is far more likely to be one of the many false alarms than one of the few true cases.
The lesson generalises well beyond medicine: when the thing you are looking for is rare, even an accurate detector produces mostly false positives. This is exactly why fraud detection, security alerting and rare-disease screening all drown in false alarms, and why it is a structural property rather than a fixable bug.
Statistics: is the difference real?
You compare two models. Model A scores 87.2%, model B scores 88.1%. Is B better?
You cannot tell from those numbers alone, and this is the most commonly skipped question in applied machine learning.
If the test set has 200 examples, a 0.9-point difference is under two examples. Reshuffle the split and the ranking may reverse. If the test set has 100,000 examples, the same gap is probably a real effect.
What to hold on to:
Sample size governs how much you can conclude. Small test sets produce differences that are pure noise. A metric quoted without the size of the set it was measured on is not interpretable.
Variance is information. Running an experiment once gives you one number. Running it five times with different random seeds gives you a range, and the range often overlaps between two models you were about to declare different. Report the range.
Sampling bias survives any amount of data. If your training data came only from one country, one season or one demographic, more of it does not fix the gap. It makes the model more confidently narrow.
Correlation is not causation, and models only ever see correlation. A model that learns "customers who contact support cancel more often" has found a correlation. Preventing customers from contacting support will not improve retention. Every model is a correlation machine, and reading causation into its features is the most expensive mistake made with them.
A model is only valid on data that resembles what it was trained on. When reality drifts — new customer behaviour, new fraud patterns, a changed product — the model does not warn you. It keeps returning confident answers computed from a world that no longer exists. Monitoring for this is the whole subject of MLOps.
The distributions worth recognising
You do not need to memorise formulas, but seeing these names and knowing what shape they describe helps enormously when reading.
- Normal (Gaussian): the bell curve. Heights, measurement errors, sums of many small independent effects. Appears everywhere because of the central limit theorem.
- Bernoulli: a single yes-or-no event. What every binary classifier is modelling.
- Binomial: how many successes in a fixed number of independent tries.
- Poisson: how many rare events in a fixed window. Arrivals, faults, clicks.
- Uniform: every outcome equally likely. Where random initialisation starts.
- Power law: a few enormous values and a very long tail. City sizes, word frequencies, wealth. Crucially, averages are misleading for these, and a great deal of real data is power-law shaped.
In three sentences
A model outputs a distribution rather than a label, and the confidence it carries is information you should use to route uncertain cases to a human rather than discard. The decision threshold is a business judgement about whether false alarms or misses cost more, which is why accuracy is meaningless on imbalanced data and precision and recall are not. Statistics is what tells you whether a difference between two models is real, and the discipline it enforces — sample sizes, repeated runs, and never reading causation into a correlation — is what separates a defensible result from a confident guess.