Module 7 — Probability, independence and conditional probability

Machine learning is, at bottom, reasoning under uncertainty: a classifier doesn't say "it's a cat" but "92% chance it's a cat." Probability is the language of that uncertainty. This module lays the foundations, aimed at what actually helps you understand models.

Probability: a measure between 0 and 1

A probability quantifies the likelihood of an event, from 0 (impossible) to 1 (certain). Two readings coexist and meet in practice:

• Frequentist: the proportion observed over many repetitions (a fair coin lands on heads about 1 in 2 times).
• Bayesian: a degree of belief, revisable in light of new data — module 8's view.

In learning, the output of a classification model is a probability, and reading it correctly changes everything about interpreting a prediction.

Joint probability and independence

The joint probability $P(A \text{ and } B)$ is that two events occur together. Two events are independent if one carries no information about the other; then, and only then:

$$P(A \text{ and } B) = P(A) \cdot P(B)$$

Two successive dice rolls are independent: the first result says nothing about the second. By contrast, "it's raining" and "the ground is wet" are not independent at all. Telling the two cases apart is crucial: wrongly assuming independence is a major error source — but it is also, sometimes, a fruitful simplification, as in the "naive" Bayes classifier of module 8.

Conditional probability: the heart of inference

The conditional probability $P(A \mid B)$ reads "probability of A given that B is true." It is the most important notion of the module, because predicting is exactly computing a conditional probability: $P(\text{class} \mid \text{observed features})$.

$$P(A \mid B) = \frac{P(A \text{ and } B)}{P(B)}$$

A medical example makes the idea concrete. The probability that a patient is ill, given that the test is positive, is not the same as the raw probability of being ill: the information "positive test" has updated our estimate. This re-evaluation in light of an observation is the central mechanism of all predictive reasoning.

# On data: P(purchase | saw_the_ad)
saw_ad = df[df["saw_ad"] == 1]
p_purchase_given_ad = saw_ad["purchased"].mean()

A classic pitfall: conditional probability is not symmetric

$P(A \mid B)$ and $P(B \mid A)$ are different, and confusing them leads to serious errors. $P(\text{positive test} \mid \text{ill})$ — the reliability of the test — may be 99%, while $P(\text{ill} \mid \text{positive test})$ — what the patient cares about — may be only 30% if the disease is rare. The exact link between these two quantities is precisely Bayes' theorem, the subject of the next module.

Why this pervades all evaluation

The metrics from the introductory course — precision, recall — are conditional probabilities in disguise. Recall is $P(\text{detected} \mid \text{actually positive})$; precision is $P(\text{actually positive} \mid \text{detected})$. They are the two directions of conditioning, which is why they differ and never reduce to a single number.

Summary

• A probability measures uncertainty between 0 and 1; a classifier's output is a probability you must know how to interpret.
• Two events are independent when $P(A \text{ and } B) = P(A)P(B)$; wrongly assuming it is a frequent error.
• Conditional probability $P(A \mid B)$ is the heart of prediction: to classify is to estimate $P(\text{class} \mid \text{features})$.
• It is not symmetric: $P(A \mid B) \neq P(B \mid A)$; confusing the two leads to serious mistakes that Bayes' theorem dispels.

Next module: Bayes' theorem, the formula linking the two directions of conditioning and underpinning a whole branch of AI.