ANOVA & t-Test Assumptions

Every test you've met comes with fine print: conditions that must roughly hold for its p-values to mean what they claim. Ignore them and the test still spits out a number; it's just the wrong number. The good news: the main assumptions are few, and most are easy to check by eye.

The big three

• Normality. The data (or, more precisely, the sampling distribution of the mean) is roughly bell-shaped. Thanks to the CLT, this matters less as samples grow.
• Homogeneity of variance. The groups you're comparing have similar spread. t-tests and ANOVA assume it (Welch's versions relax it).
• Independence. Observations don't influence each other. This is the unforgiving one: no clever correction saves a test when independence is violated by, say, repeated measurements treated as separate people.

Those three cover tests that compare group means. The moment you fit a line, a fourth joins them: linearity, the assumption that the relationship you are modeling really is straight. It is the one no diagnostic on this page can catch. Curved data can be perfectly normal and perfectly equal-variance, and a straight line through it will still give you a slope that describes nothing. Seeing it takes a residual plot, which is where regression diagnostics picks the story up.

Checking normality with a Q-Q plot

A histogram hints at shape, but the sharpest tool is the Q-Q plot. It plots your sorted data against the values you'd expect if the data were perfectly normal. If the points fall along a straight line, the data is behaving like a normal sample. Systematic curves away from the line reveal exactly how it deviates. Switch the underlying distribution below and learn to read the signatures.

How crooked does normal data look?

The playground shows four clean signatures, and real samples are messier than any of them. Before you can call a Q-Q plot bent, you need to know how bent a genuinely normal one gets. At the ends, very.

Draw 20 values from a perfect normal distribution and plot them. The highest point is drawn at 1.87 standard deviations, because that's where the twentieth of twenty expected quantiles sits. Where it actually lands is another matter: across 200,000 simulated normal samples it fell anywhere between 1.29 and 2.71 in 95% of them. The point in the middle of that same plot stayed inside [−0.38, 0.26]. So the ends of a normal Q-Q plot wobble about 2.2 times as much as the middle, and the faint S-curve your eye keeps reporting is mostly the ends doing what ends do.

Now draw a band around the line wide enough to hold 95% of normal samples at each position separately, and count the normal samples that still poke out of it somewhere: 49% at n = 20, and 66% at n = 50. Half of all perfectly normal datasets have a point outside the band, because a plot with twenty chances to escape a 95% band usually takes one.

Judge the middle, forgive the ends, and read the plot for a systematic bend rather than for stray points. When the call really matters, stop judging by eye and draw the band: R's car::qqPlot(x) shades it for you, and by hand it's a short loop. Simulate a few hundred normal samples at your own n, keep the 2.5th and 97.5th percentile at each position, and see whether your data leaves the envelope in a run rather than at one lonely corner.

How much do violations matter?

Not all violations are equal. t-tests and ANOVA are surprisingly robust to mild non-normality, especially with larger samples, where the CLT does the heavy lifting. They're more sensitive to very unequal variances (use Welch's correction) and badly thrown off by outliers in small samples. The descriptives calculator will flag those outliers and print the skewness for any data you paste into it. And nothing rescues violated independence — that one you fix in the study design, not the analysis.

When an assumption really is broken

• Transform the data (a log transform often tames right-skew).
• Use a robust variant (Welch's t, trimmed means).
• Switch to a non-parametric test that makes no normality assumption at all.

The harder judgment is the one before that: deciding whether a flagged point is a violation or just a person. The complete worked project works through four such points on a real dataset and keeps all four.

Problem 17 of the practice problems hands you two clinics whose SDs are 3.1 and 6.9 and asks you to decide, before looking at either answer, whether to pool. Welch and Student disagree by more than you might expect on the same numbers.