Probability Basics
Probability is how we put a number on uncertainty. A probability is just a value between 0 (impossible) and 1 (certain) that says how likely something is. The most useful way to think about it: a probability is the long-run frequency of an event if you could repeat the situation over and over.
Probability as long-run frequency
Say a coin has a 0.5 chance of heads. That doesn't promise heads on any particular flip, or even 5 heads in 10. It's a statement about the long run: flip it enough times and the proportion of heads will close in on 0.5. This is the Law of Large Numbers, and it's the bridge between abstract probability and actual data. It has a close relative you meet later: the law says where the proportion settles, while the Central Limit Theorem describes the shape of the wobble on the way there.
🎮 Law of Large Numbers
Set how biased the coin is, then flip. The jagged line is the running proportion of heads; the dashed line is the true probability it's chasing.
Two things to notice: early on, the proportion swings wildly — a handful of flips tells you almost nothing. As the flips pile up, the line stops wandering and hugs the true value. (And note: even a "fair" coin spends most of its time slightly off 0.5 — it converges, but it never has to land exactly on it.)
A few rules you'll lean on
- Range: every probability is between 0 and 1, and the probabilities of all possible outcomes sum to 1.
- Complement: P(not A) = 1 − P(A). The chance of not rolling a 6 is 1 − 1/6 = 5/6.
- Independent "and": if two events don't influence each other, P(A and B) = P(A) × P(B). Two fair coins both heads: ½ × ½ = ¼.
- Mutually exclusive "or": if two events can't both happen, P(A or B) = P(A) + P(B). Rolling a 1 or a 2: 1/6 + 1/6 = 1/3.
Watch out for "independent." Multiplying only works when events truly don't affect each other. Drawing cards without replacement, or events that share a common cause, break independence — and most real-world probability mistakes come from assuming it when it doesn't hold.
When one event tells you about another
The rule for events that do affect each other is conditional probability, written P(A | B) and read "the probability of A given B." It asks a narrower question than P(A): not "how common is this in everyone" but "how common is this among the people who are also B." Those two numbers can differ enormously, and confusing them is behind a great many bad statistics. P(cycles to work) across a whole company might be .35 while P(cycles to work | under 30) is .48, and the gap between them is precisely the association the rest of Stats 2 is built to test. Independence is just the special case where the gap is zero: P(A | B) = P(A), so learning B tells you nothing about A. Turning a conditional probability around, from P(test positive | ill) to the P(ill | test positive) you usually care about, is what Bayes' theorem is for.
Why it matters: every p-value, confidence level, and sampling distribution is a probability statement. "5% of intervals miss" and "p < 0.05" only mean something because of the long-run logic you just watched unfold.
Problem 5 of the practice problems reads all four kinds of probability off one 2 × 2 table of who cycles to work, and finishes by asking whether age band and cycling are independent at all.
Common questions
What is the difference between independent and mutually exclusive events?
Independent events don't influence each other — knowing one happened tells you nothing about the other (two separate coin flips), and P(A and B) = P(A) × P(B). Mutually exclusive events can't both happen (one die roll can't be both a 1 and a 2), so P(A or B) = P(A) + P(B). They're near-opposites: mutually exclusive events are maximally dependent, since one happening rules the other out.
What is the gambler's fallacy?
The belief that after a run of heads, tails is "due." It isn't — each flip is independent, and the coin has no memory. The Law of Large Numbers says the long-run proportion converges to the truth; it never promises short-run correction. The lopsided early flips just get diluted by the thousands that follow.
A test is 99% accurate and I tested positive. Am I 99% likely to have the disease?
No, and the gap is startling. Suppose 1 person in 1,000 has the condition, and the test catches 99% of real cases while wrongly flagging 1% of healthy people. Out of 100,000 people, 100 are ill and about 99 of them test positive; the other 99,900 are healthy and about 999 of them also test positive. So roughly 1,098 positive results contain 99 real cases, and P(ill | positive) is about 9%, not 99%. The test's accuracy answers P(positive | ill), and you wanted P(ill | positive), which is a different conditional probability. Nothing is wrong with the test; the base rate is simply so low that false positives outnumber true ones. This reversal is exactly what Bayes' theorem computes, and it is the reason screening programs report positive predictive value rather than accuracy.