Missing Data & Imputation

Real datasets have holes: participants skip questions, sensors fail, people drop out. The reflex (analyze whoever's left, or plug gaps with the average) feels harmless. But whether it is harmless depends entirely on why the data are missing. Statisticians sort that "why" into three mechanisms, and the difference between them decides whether your results are fine, slightly noisy, or systematically wrong.

Three ways data go missing

• MCAR: Missing Completely At Random. The holes are pure chance, unrelated to anything (a dropped test tube). Analyses of the remaining data are unbiased; you just lose precision.
• MAR: Missing At Random (a terrible name). Missingness depends on other observed variables, not on the missing value itself: for example, students with heavy course loads (observed) skip your exam more often. Complete-case analysis can be biased, but the observed data contain enough information to correct it.
• MNAR: Missing Not At Random. Missingness depends on the missing value itself — the students who'd have scored worst are precisely the ones who didn't show. This is the dangerous one: the observed data alone can't fully fix it.

Why mean imputation is a trap

Replacing every hole with the observed average looks tidy, but you've invented data with zero variability sitting exactly on the mean. The imputed sample's standard deviation shrinks, every correlation weakens (all those flat points pull r toward 0), and, worst of all, your software now believes it has a full n, so standard errors and p-values come out overconfident. The visualization makes this visible: flip to mean imputation and watch the SD and r readouts sag while the mean stays put.

What your software calls all this

Complete-case analysis goes by another name in every menu you will actually click: listwise deletion, or "exclude cases listwise". Its neighbor in the same dialog, pairwise deletion, keeps a case for every calculation it has the values for, so a correlation matrix ends up built from a different subset of people in every cell. It keeps more data than listwise does, and two things go with it. You no longer have one honest n to report. And the matrix is no longer guaranteed to be internally consistent: with 35% of values missing at random, a pairwise correlation matrix comes out mathematically impossible in roughly one simulated dataset in forty, carrying a negative eigenvalue that no real correlation matrix can have, at which point factor analysis or regression either refuses to run or runs on nonsense.

Neither is a modern default. Both are worth recognizing on sight, because they are what the software offers first.

What to do instead

The modern default is multiple imputation: fill each hole several times with plausible values drawn from a model of the data (spread included!), analyze each completed dataset, and pool the results, so the extra uncertainty from the holes flows honestly into your standard errors. Maximum-likelihood approaches (like FIML, standard in mixed models software) get the same job done. Both are valid under MCAR and MAR; under MNAR nothing is automatic and sensitivity analysis becomes your friend.

Two habits upstream make all of this easier. Record missingness in a way that survives the trip to your analysis software, which means one documented code per reason in the codebook rather than a blank cell that could mean anything, and keep the decision about who gets dropped inside a cleaning script where it can be read and re-run instead of in a spreadsheet where it cannot.

Problem 30 of the practice problems is this lesson with the stakes attached: 24 of 180 trial participants never came back for the final weigh-in, 18 of them from the intervention arm, and the analyst simply dropped them.