Conditional Probability
Statistics & ProbabilityConditional probability is the probability of an event occurring given that another event has already occurred.
Formula
P(B|A) = \dfrac{P(A \text{ and } B)}{P(A)}
Definition
Conditional probability is the chance that something happens given that we already know something else happened. The extra information changes the probability.
Example
What is the probability of rolling a $4$ given that you already know the roll was even? The even outcomes are $\{2,4,6\}$, and only one is a $4$. So $P(4 \mid \text{even}) = 1/3$.
Key Insight
Conditional probability is about updating your prediction with new information. "Given that..." is the key phrase that signals conditional probability.
Definition
The conditional probability of $B$ given $A$ is $P(B|A) = P(A \text{ and } B)/P(A)$, defined when $P(A) > 0$. It represents the probability of $B$ in the restricted sample space where $A$ is known to have occurred. If $P(B|A) = P(B)$, the events are independent.
Example
From a class of $30$: $18$ play sports, $12$ play music, $6$ play both. $P(\text{music} \mid \text{sports}) = P(\text{music and sports})/P(\text{sports}) = (6/30)/(18/30) = 6/18 = 1/3$. Among sports players, $1/3$ also play music.
Key Insight
Two-way tables (contingency tables) are ideal for computing conditional probabilities. Focus on the row or column corresponding to the given condition and treat it as a new sample space.
Definition
Conditional probability $P(B|A) = P(A \cap B)/P(A)$ extends to conditional expectation $E[Y|X]$, a random variable that equals $E[Y|X=x]$ for each value $x$ of $X$. The tower property $E[Y] = E[E[Y|X]]$ is fundamental in probability and statistics. Bayes' theorem follows directly: $P(A|B) = P(B|A)P(A)/P(B)$.
Example
Bayes' theorem for medical diagnosis: $P(\text{disease} \mid \text{positive test}) = P(\text{positive} \mid \text{disease})P(\text{disease})/P(\text{positive})$. With prevalence $P(\text{disease})=0.01$, sensitivity $P(\text{pos}|\text{dis})=0.95$, specificity $P(\text{neg}|\text{no dis})=0.90$: $P(\text{pos}) = 0.95(0.01) + 0.10(0.99) = 0.1085$. $P(\text{disease} \mid \text{positive}) = 0.95(0.01)/0.1085 = 0.0876$ (only $8.76\%$ despite a positive test).
Key Insight
The low base-rate paradox: even a highly accurate test has many false positives when the condition is rare, because the probability of a true positive is swamped by the large number of negatives in the population. This has profound implications for medical screening programs.