One-Way ANOVA

The t-test compares two groups. But what if you have three diets, four teaching methods, or five doses? Analysis of Variance (ANOVA) compares all the group means at once. The trick is beautiful: instead of comparing means directly, it compares variances.

Why not just run lots of t-tests?

With four groups there are six possible pairwise t-tests. Each carries a 5% false-positive risk, and those risks pile up: run enough and you're almost guaranteed a "significant" result by chance alone. ANOVA sidesteps this with a single omnibus test: one verdict on whether any of the groups differ.

The pile-up has a size, and it is worth working out once. With I groups there are I(I − 1)/2 pairs, so three groups give three tests and four groups give six. If the tests were independent and every null were true, the chance of at least one false alarm across k of them is 1 − (1 − α)^k:

1 − (1 − .05)³ = .1426    1 − (1 − .05)⁶ = .2649

Six comparisons advertised at 5% deliver something closer to 26%. Pairwise tests on the same data are not really independent, so the true rate sits a little below those figures, but the size of the problem is right and it grows fast. The omnibus test buys one verdict at one α, and post-hoc tests pay a controlled price afterwards for the comparisons you actually want.

The F-ratio: signal vs. noise, again

ANOVA splits the total variation in the data into two pieces and takes their ratio:

F = (variance between groups) / (variance within groups)

If the groups really differ, their means are spread out (big between-group variance) relative to the scatter inside each group, so F is large. If the groups are all basically the same, the between-group variance is just noise of the same size as the within-group variance, and F sits near 1. The bookkeeping underneath (each MS as SS ÷ df, the two df, and η² from the same table) is laid out on the formula sheet.

Reading the result

The F-ratio comes with two degrees of freedom, one for each half of the ratio, and together they give the p-value. As usual, p < 0.05 means "this much separation would be surprising if all the groups were really identical." If you are still designing the study rather than reading one, the power calculator runs the same numbers backwards: tell it how many groups and how big an effect you care about, and it returns the n per group you need.

Explained and unexplained variation

Sixteen seedlings, and no two of them the same height. That variation came from somewhere, and the whole method of ANOVA is to sort it into two piles.

Ask two questions about any single seedling. How far is its group's mean from the mean of everybody? That distance is what the fertilizer dose might account for. And how far is the seedling itself from its own group's mean? Nothing in the design accounts for that, because two seedlings on the same dose were treated identically. The first pile is explained, or between-group, variation: the signal. The second is unexplained, or within-group, variation: the noise, and the technical name for one of those distances is a residual.

Each question is answered against a different mean, which is the detail most hand calculations get wrong. Between-group variation is measured from the grand mean, written x̄ with no subscript: the mean of all N observations pooled together, ignoring which group they came from. Within-group variation is measured from each observation's own group mean x̄_i.

The three sums of squares

Distances either side of a mean cancel to zero by construction, so ANOVA squares them before adding, the same move the standard deviation makes for the same reason. A total of squared distances is a sum of squares, written SS, and each pile has one:

Σ (sigma) is an instruction to add up whatever follows it. In SS_G the sum runs over the I groups, and each squared distance is weighted by n_i, that group's own size, because a mean built from twenty observations speaks for twenty of them. In SS_E and SS_T the sum runs over all N observations.

The second form of SS_E is the one to reach for when a question hands you summary statistics rather than raw scores: multiply each group's variance by its own degrees of freedom, n_i − 1, and add. Every s_i here is a sample standard deviation on n_i − 1, which is what SPSS prints and what a calculator's s key returns.

These are not three separate calculations. They satisfy an identity that holds for any data at all:

SS_T = SS_G + SS_E

which is the algebra of the sentence above. Every scrap of variation is either between the groups or inside them, and the two pieces add back to the whole. It is also the cheapest arithmetic check available: work out all three, and if they do not add up, one of them is wrong.

Degrees of freedom, mean squares, and F

A sum of squares grows with the number of things summed, so on its own it says nothing about size. Dividing by the count of independent pieces of information behind it fixes that, and that count is the degrees of freedom:

df_G = I − 1    df_E = N − I    df_T = N − 1

with I the number of groups and N the total number of observations. Some books write k where this one writes I; the arithmetic is the same either way. The three add up exactly as the sums of squares do, since (I − 1) + (N − I) = N − 1, and each comes from the same argument used everywhere else: I group means free to move around a grand mean they themselves fix leaves I − 1, and each group of n_i scores around its own mean leaves n_i − 1, which adds across groups to N − I.

Divide a sum of squares by its degrees of freedom and you have a mean square, which is a variance:

MS_G = SS_G / df_G    MS_E = SS_E / df_E    F = MS_G / MS_E

and the ratio is read against F(I − 1, N − I). There is no MS_T row worth printing, because SS_T/df_T is just the variance of the whole sample and no test uses it.

MS_E is the more useful of the two once the omnibus test is over. It is the pooled estimate of within-group variance, the same pooled variance the two-sample t-test builds from two groups, generalized to I. Its square root is the pooled standard deviation,

s_p = √MS_E

and every follow-up comparison in the next section reuses it instead of recomputing a spread from the two groups in front of it. That is the practical argument for building the omnibus table even when you already know which pairs interest you: the table hands the follow-ups their error term.

One more number falls out of the same table, from two of its cells. The share of the total variation the grouping accounts for is

η² = SS_G / SS_T

read eta squared, an effect size on the same 0-to-1 scale as R², and the reason effect sizes belong in the same sentence as the F. Because it is Greek it stays upright in APA style, and because it cannot exceed 1 it takes no leading zero.

On the seedlings it boots with, the four doses are None, Low, Medium and High, and their means are 13, 17, 19 and 23 centimeters against a grand mean of 18, and the table comes out as SS_G = 208 on 3 degrees of freedom, SS_E = 48 on 12, SS_T = 256 on 15. That gives MS_G = 69.33, MS_E = 4.00, F(3, 12) = 17.33 and p = .0001, which an APA sentence reports as p < .001, with s_p = 2.00 and η² = .813. Four seedlings a group is smaller than any study you would run; it is the size that fits on a page and on an exam paper, which is the job it is doing here.

Every score is its group mean plus a residual

Write one seedling's height out in full and the partition stops being an accounting trick:

x = x̄ + (x̄_i − x̄) + (x − x̄_i)

The three terms are the grand mean, the group's effect, and the residual. Fold the first two together and the sentence gets shorter: data = fit + residual. The fitted value for any observation is simply its own group mean, which is all this model has to offer, since it knows nothing about a seedling except which dose it received. Square each term, add over all N observations, and the cross-products vanish; what survives is SS_T = SS_G + SS_E.

That is the same decomposition regression uses, with a fitted line where this has a set of group means, which is why the two procedures print the same shape of output table. Categorical predictors and dummy coding makes the connection exact, by fitting the same group means as a regression on dummies.

Two groups, and F = t²

Nothing stops you running an ANOVA on two groups, and doing it once is worth the trouble. Load the builder with the untreated and high-dose seedlings alone, eight numbers and two groups, and the table returns F(1, 6) = 60.00 with p = .0002. Run the pooled two-sample t-test on those same eight numbers and it returns t(6) = 7.746, whose two-tailed p is the same number to every digit a calculator will show you. And 7.746² = 60.00.

The identity F = t² holds whenever df_G = 1, which is no coincidence: squaring a t on v degrees of freedom gives an F on (1, v), so the two tests are one test written twice. What the t version adds is a signed difference and a confidence interval for it, which is why it stays the one to report when there are two groups. ANOVA earns its keep from three groups upward.

What you're assuming

• The groups are independent, and observations within each are independent.
• The outcome is roughly normal within each group (or samples are large).
• Homogeneity of variance, meaning the groups have similar spread. Checking that properly takes a plot and a test, and the next section covers what to run when the check fails.

When the spreads are not equal

The practice file at the foot of this lesson was built to break this assumption. study-methods.csv holds 35 students per revision method with standard deviations of 7.97, 8.86 and 11.97, and Levene's test duly objects: F(2, 102) = 3.54, p = .033. Run the ordinary ANOVA anyway and it returns F(2, 102) = 11.57, p < .001. Run Welch's ANOVA, which weights each group by its own precision rather than pooling everyone into one error term, and it returns F(2, 66.43) = 13.10, p < .001. Same story, with a fractional denominator df because the weighting has to be paid for somewhere.

The agreement is not luck, and the reason is the part the textbook warning usually omits. Unequal variances alone do surprisingly little damage; unequal variances paired with unequal group sizes do a great deal. Three groups drawn from populations that genuinely do not differ, 100,000 samples per row, testing at α = .05:

Read the middle row first. The smallest group is the noisiest one, and a test advertised at 5% false alarms delivers 16%. Flip the pairing so the big group carries the big spread and the rate drops to 2%. That direction looks like the safe one to err in, and it costs you too: a test that rejects too rarely under the null is missing real effects at the same time. The site's own dataset lands in none of these traps because its groups are the same size, and equal n is doing most of the protecting.

Welch holds near 5% in all three rows. What it costs when the variances really are equal is small and measurable: about one percentage point of power at 20 per group, about three at 10 per group. SPSS prints it under Robust Tests of Equality of Means, on a line above a second robust test called Brown–Forsythe. Be careful with that name, because two different procedures wear it. One is the robust F in that box; the other is the version of Levene's test that centers on medians instead of means, which is what R's car::leveneTest gives you by default. A significant omnibus still owes you follow-up comparisons, and those need the same correction the omnibus did: Games–Howell is Tukey's HSD rebuilt on the same principle.

For a run on real data, study-methods.csv compares three revision techniques across 105 students, with group SDs deliberately unequal so Levene's test has something to say.

Problem 13 of the practice problems builds a complete ANOVA table from fifteen raw scores, and Problem 14 carries the same data into the follow-up comparisons. Two more work the shapes an exam prefers: Problem 23 in the Stats 2 set rebuilds a whole table from three groups' means, SDs and n, and Problem 24 hands you a table with four cells missing.