ANCOVA: Controlling for Covariates
A training program looks brilliant — until you notice its students were already stronger on the pretest. ANCOVA (Analysis of Covariance) compares group means after statistically holding a covariate constant, so you compare like with like. It's the workhorse behind every "controlling for age / baseline / IQ" you've ever read.
Why compare like with like
An ordinary ANOVA asks: do the group means differ? But raw means mix two things together: the effect of the treatment and whatever differences the groups walked in with. If the treatment group started ahead on a variable that predicts the outcome (a covariate), the raw comparison flatters the treatment. If they started behind, it hides a real effect.
ANCOVA fixes this with a simple, elegant move: fit a regression line from the covariate to the outcome within each group, then compare the groups at the same covariate value (conventionally the overall mean). Those are the adjusted means, and the vertical gap between the parallel lines is the adjusted effect.
🎮 The Fair-Comparison Machine
Each dot is a person: pretest score across, outcome up. The lines are the within-group regression fits; the dashed vertical line is the overall pretest mean, and the big markers on it are the adjusted means. Give the orange group a head start and watch the naive comparison lie while ANCOVA keeps its footing. (The same people stay on screen as you drag; "New sample" redraws them.)
What ANCOVA actually does
Two jobs at once:
- It corrects for baseline imbalance. The adjusted difference is the raw difference minus the part explained by the covariate gap: adjusted = raw − slope × (covariate gap between groups). Set the head start to +10 with a real effect of 0 and watch the naive test declare victory while ANCOVA correctly shrugs.
- It soaks up noise. Even with zero imbalance (perfect randomization), the covariate explains outcome variation that would otherwise sit in the error term. Removing it shrinks the residual noise, so the same effect gets a smaller p-value — free power. Try it: head start 0, effect 5, slope high, and compare the two p-values.
The two-slider magic trick: set the true effect to 0 and the head start to +10. The naive t-test "finds" an effect that is pure baseline difference, while ANCOVA reports ≈ 0. Now flip the head start to −10 with a true effect of 6: the naive test misses a real effect that ANCOVA recovers. Same data-generating world, opposite naive conclusions. That's why baseline adjustment matters.
The assumptions that make it honest
- Linearity: the covariate–outcome relationship should be roughly a straight line within each group.
- Homogeneity of regression slopes: the group lines must be parallel: the covariate's effect is the same in every group. If treatment changes the slope itself, a single "adjusted difference" doesn't exist; fit the interaction model instead and report how the effect varies.
- The covariate is measured before (and unaffected by) treatment. Adjusting for a variable the treatment itself moved throws away part of the effect you're trying to measure.
- Plus the usual suspects from ANOVA: independent observations, roughly normal residuals, similar residual spread.
"Controlling for" is not magic. In a randomized experiment, ANCOVA buys precision. In observational data it only removes the confounding carried by the covariates you measured, never the ones you didn't. And adjusting for a post-treatment variable or a collider can actively create bias. When causality is the question, draw the DAG first, then decide what belongs in the model.
How big is the adjusted difference?
An F and a p say the adjusted difference is unlikely to be zero. Neither says whether it is worth anything. The usual companion is partial eta squared, written ηp²: the share of outcome variation the grouping factor explains once the covariate has taken its own share out of the denominator.
ηp² = SSgroup ÷ (SSgroup + SSerror)
You can also read it off the F table without any sums of squares, since ηp² = F·df₁ ÷ (F·df₁ + df₂). An ANCOVA reported as F(1, 57) = 6.84 therefore carries ηp² = .11. Both SPSS and JASP print it under "Estimates of effect size", and APA writes it upright with no leading zero.
The word partial earns its place. The covariate's variance has already left the denominator, so when randomization has balanced that covariate, ηp² comes out larger than the plain η² an unadjusted ANOVA would give on the same outcome; where the groups differed on the covariate to begin with, the adjustment shrinks the numerator too and it can go the other way. Either way the two numbers answer different questions and shouldn't be pooled or compared as though they were one. State which you are reporting and which covariates were in the model, and quote the adjusted difference in the outcome's own units as well, because that is the number a reader will remember.
One family: ANCOVA, ANOVA, regression
Under the hood ANCOVA is just multiple regression with a dummy-coded group and a continuous covariate: outcome = b₀ + b₁·group + b₂·covariate. The b₁ coefficient is the adjusted difference. ANOVA, ANCOVA, and regression are one linear model wearing three outfits. And when you have several outcome variables to adjust at once, the same idea scales up to MANCOVA.
Why it matters: almost every serious group comparison in psychology, medicine, and education is covariate-adjusted: pre/post designs, clinical baselines, demographic controls. Knowing what the adjustment does (and what it can't do) is the difference between reading results and being fooled by them. For a design of your own, Plan My Analysis lays out the assumptions to check and the write-up before you collect anything, and Choosing Statistics for Your Dissertation answers the version of this that supervisors actually hear: "but I need to control for age".
The adjustment is four lines of arithmetic once you have the slope. Problem 24 in the practice problems works it through on a reading program whose 6.4-point advantage turns out to be mostly a head start.
Common questions
What is the difference between ANOVA and ANCOVA?
ANOVA compares raw group means. ANCOVA first removes the part of the outcome explained by a continuous covariate (pretest score, age, baseline severity), then compares the adjusted means. Two payoffs: it corrects for baseline imbalance between groups, and it soaks up noise — often shrinking p-values even in perfectly randomized experiments.
Can ANCOVA fix pre-existing group differences in observational studies?
Only partially, and this deserves respect: adjustment removes confounding carried by the covariates you measured — never the ones you didn't. Worse, when groups differ substantially on the covariate, adjusted comparisons extrapolate into covariate territory where one group has no data (this is the heart of Lord's paradox). ANCOVA sharpens randomized experiments; in observational data it's an assumption-laden estimate, not a magic equalizer.
Should I analyze change scores (post minus pre) instead of using ANCOVA?
In a randomized experiment ANCOVA is the better default, for reasons that are arithmetic rather than taste. A change-score analysis is ANCOVA with the pretest slope forced to 1, while ANCOVA estimates that slope from the data. Whenever the true pre-post correlation r is anything below 1, the estimated slope strips out more noise, and the standard error of the treatment effect comes out √((1 + r) / 2) times the change-score version's: roughly 13% smaller when pre and post correlate .5, and 30% smaller when they hardly correlate at all. The two agree exactly only at r = 1. Outside a randomized design they can disagree about direction as well as precision, which is what Lord's paradox describes.