Chi-Square Tests
Everything so far has compared means of numeric data. But what about counts: how many people picked option A vs. B, recovered vs. didn't, voted left vs. right? The chi-square (χ²) test works with frequencies in categories, and its most common job is asking: are two categorical variables associated, or independent?
Observed vs. expected
The whole test rests on one comparison. You have the observed counts (what actually happened). Then you compute the expected counts: what you'd see if the two variables were completely unrelated (each cell's expected count is just its row total × column total ÷ grand total). The bigger the gap between observed and expected, the more evidence of a real association:
χ² = Σ (observed − expected)² / expected
The Σ (sigma) is an instruction to add: work the fraction out for one cell, then do it for every cell and total them.
That formula, the expected-count rule, the df and Cramér's V are four consecutive rows on the printable formula sheet, worth having beside you the first few times.
🎮 Contingency Table Explorer
A treatment trial. Each bar shows the recovery split for one group. When the two splits match the dashed "no-association" line, χ² is tiny. Pull them apart and watch χ² climb.
Reading the result
The degrees of freedom for a table are (rows − 1) × (columns − 1), just 1 for our 2×2. A χ² statistic far from zero, with p < 0.05, means the observed counts would be surprising if the variables were truly independent, so you conclude they're associated. In the trial: treatment and recovery are related.
Association, not amount. Like correlation, a significant χ² tells you a relationship exists, not how strong it is. For strength, report an effect size such as Cramér's V or the odds ratio alongside the test; the effect-size converter will turn an odds ratio into the more familiar d or r if you need to compare it with the rest of your results.
Two flavors of chi-square
- Test of independence is the one above: are two categorical variables related? (Treatment × outcome, gender × preference.)
- Goodness-of-fit: does one categorical variable match an expected distribution? (Is this die fair? Do customers pick the four flavors equally?)
What you're assuming
- Counts are independent: each observation falls in exactly one cell, and one person's category doesn't affect another's.
- Expected counts are large enough. A common rule is every expected count ≥ 5. For small 2×2 tables, use Yates' continuity correction or Fisher's exact test instead. Counts are cheap to collect and easy to under-collect, so if the study is still ahead of you, the power calculator will tell you how many observations a given association needs.
When the same person appears twice
Two different meanings of "independent" meet in this test, and only one of them is the null hypothesis. The test asks whether the two variables are independent. The assumption above is about the observations: sixty people should produce sixty entries, not a hundred and twenty.
Sixty students say whether they feel confident reading a results section, once before a workshop and once after. Stack the two occasions and you have a 2 × 2 table of 120 answers, which looks like a perfectly ordinary chi-square problem. It is not one. Each student sits in that table twice, so the table has 60 independent units, and a test built on 120 will describe a study nobody ran.
The fix is to make the person the row. Cross each student's before-answer with their own after-answer and every student lands in exactly one of four cells: yes both times, no both times, or one of the two switchers. The students who answered the same way twice tell you nothing about whether the workshop changed anything, so McNemar's test throws them out and works only on the switchers, which textbooks call the discordant pairs:
χ² = (b − c)² ⁄ (b + c), with 1 degree of freedom
where b and c are the two off-diagonal counts. Suppose 18 students went from no to yes and 6 went the other way. Then χ² = (18 − 6)² / 24 = 6.00 and p = .014. SPSS and R apply a continuity correction by default, giving 5.04 and p = .025; with only 24 switchers you can skip the approximation entirely and ask the binomial directly, which returns p = .023. All three say the same thing, and the exact one is the one to quote when b + c is small.
What does stacking the occasions actually cost? Simulating 60 students whose two answers correlate at .6, with no real change at all, the ordinary chi-square on the 120 rows rejects about 1% of the time rather than 5%. It is too cautious, not too eager, which catches most people out. Give the same students a genuine shift, from 60% confident to 78%, and McNemar finds it 78% of the time against the stacked test's 60%. Break the link between the two answers so that the pairing carries no information and the two tests land on top of each other, at 58% each. Pairing is worth using precisely when there is a pairing to use, and ignoring it here wastes participants rather than manufacturing findings.
Three or more occasions on the same people follow the same logic under a different name, Cochran's Q, which generalizes McNemar to k repeated yes/no measurements. The printable assumptions poster names McNemar as the fix for this row; the reason it is a fix, and what happens when the repetition is people inside schools rather than occasions inside people, is mixed & multilevel models.
Why it matters: survey responses, A/B test conversions, medical outcomes, demographic breakdowns: categorical data is everywhere, and chi-square is the workhorse for testing whether the categories hang together. Its neighbors in the association family, and when each one applies, sit in the third panel of the printable test-chooser poster.
Problem 19 in the practice problems works a 2 × 3 table from expected counts through to Cramér's V, and shows how to find the cell that is doing the damage.
Common questions
My chi-square is significant. Which cell is responsible?
The test itself will not tell you, because it pools every cell into one number. Ask the table for adjusted standardized residuals, which SPSS offers under Crosstabs → Cells and R returns as chisq.test(tab)$stdres. Each one is roughly a z-score for its cell under independence, so a value beyond about ±2 marks a cell holding noticeably more or fewer cases than independence predicts, and the sign tells you which direction. Read the pattern before you write the sentence: a significant χ² on a 3 × 4 table usually comes from one or two cells, not from the whole table drifting at once. With many cells, treat the residuals as exploratory rather than as a stack of formal tests.
What if my expected counts are less than 5?
The χ² p-value is an approximation that degrades with small expected counts. The standard rule: all (or at least 80% of) expected counts should be ≥ 5. Below that, use Fisher's exact test for 2×2 tables (exact, no approximation) — or collapse sparse categories together when it makes conceptual sense.
Can a chi-square test tell me how strong the association is?
χ² itself can't; it grows with sample size, so a huge study can produce an enormous χ² from a trivial association. Pair the test with an effect size: Cramér's V (0 = none, 1 = perfect) for general tables, or the odds ratio for 2×2 tables, which our effect-size converter can translate into other metrics.