One-Way ANOVA

The t-test compares two groups. But what if you have three diets, four teaching methods, or five doses? Analysis of Variance (ANOVA) compares all the group means at once, and it does so by comparing two variances.

Why not just run lots of t-tests?

With four groups there are six possible pairwise t-tests. Each carries a 5% false-positive risk, and those risks pile up: run enough and you're almost guaranteed a "significant" result by chance alone. ANOVA sidesteps this with a single omnibus test: one verdict on whether any of the groups differ.

With I groups there are I(I − 1)/2 pairs, so three groups give three tests and four groups give six. If the tests were independent and every null were true, the chance of at least one false alarm across k of them is 1 − (1 − α)^k:

1 − (1 − .05)³ = .1426    1 − (1 − .05)⁶ = .2649

Six comparisons, each run at 5%, give a false-alarm rate closer to 26%. Pairwise tests on the same data are not fully independent, so the true rate is a little below those figures, but the size of the problem is right and it grows fast. The omnibus test gives one verdict at one α, and post-hoc tests then make the comparisons you actually want with the error rate under control.

The F-ratio: signal vs. noise, again

ANOVA splits the total variation in the data into two pieces and takes their ratio:

F = (variance between groups) / (variance within groups)

If the population means differ, the group means are spread out (big between-group variance) relative to the scatter inside each group, so F is large. If the population means are all equal, the between-group variance is just noise of the same size as the within-group variance, and F is near 1. The bookkeeping underneath (each MS as SS ÷ df, the two df, and η² from the same table) is laid out on the formula sheet.

Reading the result

The F-ratio comes with two degrees of freedom, one for each half of the ratio, and together they give the p-value. As usual, p < 0.05 means "this much separation would be surprising if all the groups were identical." If you are designing a study, the power calculator runs the same numbers backwards. Tell it how many groups you have and how big an effect you care about, and it returns the n per group you need.

Explained and unexplained variation

Take sixteen seedlings, no two of them the same height. ANOVA sorts that variation into two piles.

Ask two questions about any single seedling. How far is its group's mean from the mean of everybody? The fertilizer dose might account for that distance. And how far is the seedling itself from its own group's mean? Nothing in the design accounts for that, because two seedlings on the same dose were treated identically. The first pile is explained, or between-group, variation: the signal. The second is unexplained, or within-group, variation: the noise. Each of those distances is called a residual.

The two questions use different means, and hand calculations often mix them up. Between-group variation is measured from the grand mean, written x̄ with no subscript: the mean of all N observations pooled together, ignoring which group they came from. Within-group variation is measured from each observation's own group mean x̄_i.

The three sums of squares

Distances on either side of a mean always add to zero, so ANOVA squares them before adding, as the standard deviation does. A total of squared distances is a sum of squares, written SS, and each pile has one:

Σ (sigma) is an instruction to add up whatever follows it. In SS_G the sum runs over the I groups, and each squared distance is weighted by n_i, that group's own size, because a mean built from twenty observations represents twenty of them. In SS_E and SS_T the sum runs over all N observations.

Use the second form of SS_E when a question gives you only summary statistics: multiply each group's variance by its own degrees of freedom, n_i − 1, and add. Every s_i here is a sample standard deviation on n_i − 1. SPSS prints that version, and so does a calculator's s key.

The three are linked by an identity that holds for any data:

SS_T = SS_G + SS_E

Every bit of variation is either between the groups or inside them, and the two pieces add back to the whole. The identity is also a quick arithmetic check: work out all three, and if they do not add up, one of them is wrong.

Degrees of freedom, mean squares, and F

A sum of squares grows with the number of things summed, so by itself it does not tell you how large the variation is. To fix that, divide by the number of independent pieces of information behind it. That number is the degrees of freedom:

df_G = I − 1    df_E = N − I    df_T = N − 1

Here I is the number of groups and N the total number of observations. Some books write k where this one writes I; the arithmetic is the same. The three add up just as the sums of squares do, since (I − 1) + (N − I) = N − 1. Each comes from the usual argument. The I group means vary around a grand mean that they themselves determine, which leaves I − 1. Each group of n_i scores varies around its own mean, which leaves n_i − 1, and those add across groups to N − I.

Divide a sum of squares by its degrees of freedom and you have a mean square, which is a variance:

MS_G = SS_G / df_G    MS_E = SS_E / df_E    F = MS_G / MS_E

The ratio is read against F(I − 1, N − I). ANOVA tables print no MS_T, because SS_T/df_T is just the variance of the whole sample and no test uses it.

MS_E is the more useful of the two once the omnibus test is over. It is the pooled estimate of within-group variance, the same pooled variance the two-sample t-test builds from two groups, generalized to I. Its square root is the pooled standard deviation,

s_p = √MS_E

Every follow-up comparison in the next section reuses it, so none of them recomputes a spread from only the two groups being compared. That is a practical reason to build the omnibus table even when you already know which pairs interest you: the follow-up tests take their error term from it.

Two cells of the same table give one more number. The share of the total variation that the grouping accounts for is

η² = SS_G / SS_T

It is read eta squared, an effect size on the same 0-to-1 scale as R². Effect sizes like this one belong in the same sentence as the F, because the F alone does not say how large the differences are. Because η² is Greek it stays upright in APA style, and because it cannot exceed 1 it takes no leading zero.

On the starting data, the four doses are None, Low, Medium and High. Their means are 13, 17, 19 and 23 centimeters, and the grand mean is 18. The table comes out as SS_G = 208 on 3 degrees of freedom, SS_E = 48 on 12, and SS_T = 256 on 15. That gives MS_G = 69.33, MS_E = 4.00, F(3, 12) = 17.33 and p = .0001, which an APA sentence reports as p < .001, with s_p = 2.00 and η² = .813. Four seedlings a group is smaller than any real study, but it fits on a page and on an exam paper.

Every score is its group mean plus a residual

Writing out one seedling's height in full shows where the partition comes from:

x = x̄ + (x̄_i − x̄) + (x − x̄_i)

The three terms are the grand mean, the group's effect, and the residual. Combine the first two and you get data = fit + residual. The fitted value for any observation is its own group mean. The model can do no better, because the only thing it uses about a seedling is which dose it received. Square each term and add over all N observations. The cross-products add to zero, and what is left is SS_T = SS_G + SS_E.

Regression uses the same decomposition, with a fitted line in place of the group means, so the two procedures print the same shape of output table. Categorical predictors and dummy coding makes the connection exact by fitting the same group means as a regression on dummies.

Two groups, and F = t²

You can run an ANOVA on two groups, and it is useful to do it once. Load the builder with the untreated and high-dose seedlings alone (eight numbers and two groups), and the table gives F(1, 6) = 60.00 with p = .0002. The pooled two-sample t-test on the same eight numbers gives t(6) = 7.746, and its two-tailed p is identical to every digit a calculator shows. And 7.746² = 60.00.

The identity F = t² holds whenever df_G = 1. Squaring a t on v degrees of freedom gives an F on (1, v), so with two groups the two tests are the same test. The t version also gives a signed difference and a confidence interval for it, so it is the one to report when there are two groups. ANOVA is needed from three groups upward.

What you're assuming

• The groups are independent, and observations within each are independent.
• The outcome is roughly normal within each group (or samples are large).
• Homogeneity of variance, meaning the groups have similar spread. Checking that properly takes a plot and a test, and the next section covers what to run when the check fails.

When the spreads are not equal

The practice file at the foot of this lesson breaks this assumption on purpose. The file study-methods.csv holds 35 students per revision method, with standard deviations of 7.97, 8.86 and 11.97, and Levene's test is significant: F(2, 102) = 3.54, p = .033. The ordinary ANOVA gives F(2, 102) = 11.57, p < .001. Welch's ANOVA weights each group by its own precision rather than pooling everyone into one error term, and it gives F(2, 66.43) = 13.10, p < .001. The conclusion is the same. Welch's denominator df is fractional, and smaller, because it is estimated from the group variances.

The two agree mainly because the three groups are the same size. Unequal variances alone do little damage. Unequal variances combined with unequal group sizes do a great deal. The table shows three groups drawn from populations with equal means, 100,000 simulated samples per row, tested at α = .05:

Start with the middle row. The smallest group is the noisiest one, and a test set at 5% gives 16% false alarms. Reverse the pairing so the largest group has the largest spread, and the rate drops to 2%. That looks like the safe direction to be wrong in, but it has a cost too: a test that rejects too rarely under the null also misses real effects.

Welch stays near 5% in all three rows. When the variances are in fact equal, it loses a little power: about one percentage point at 20 per group, and about three at 10 per group. SPSS prints it under Robust Tests of Equality of Means, on a line above a second robust test called Brown–Forsythe. Be careful with that name, because two different procedures use it. One is the robust F in that box. The other is the version of Levene's test that centers on medians instead of means, and it is what R's car::leveneTest gives you by default. A significant omnibus test still needs follow-up comparisons, and those need the same correction: Games–Howell is Tukey's HSD rebuilt on the same principle.

For practice on a larger dataset, study-methods.csv compares three revision techniques across 105 students, with group SDs made unequal on purpose so that Levene's test flags them. If this ANOVA is part of a larger project, see One Question, an ANOVA and a Regression. It takes the same test from the research question to the write-up, on a file that also has two control variables.

Problem 34 of the practice problems builds a complete ANOVA table from fifteen raw scores, and Problem 35 carries the same data into the follow-up comparisons. Two more use the formats exams prefer. Problem 44 in the Stats 2 set rebuilds a whole table from three groups' means, SDs and n, and Problem 45 gives you a table with four cells missing.