Factorial ANOVA (Two-Way)

Real situations rarely involve just one factor. Does a drug's effect depend on dose and on age? Does a teaching method work differently for beginners than experts? A factorial ANOVA tests two (or more) factors at once and reveals something no single-factor study ever could: the interaction.

Factors, levels, and cells

Three words carry most of the notation. A factor is a categorical variable the study manipulates or classifies people by. Its levels are the categories it takes, so a factor called Dose might have the levels None, Low and High. Cross two factors, pairing every level of one with every level of the other, and each pairing is a cell.

The design is then named by its levels, multiplied. Two levels of each factor is a 2 × 2 with four cells. Three levels of each is a 3 × 3 with nine. Four doses crossed with two age bands is a 4 × 2 with eight. Read the notation and you know how many factors there are, from the count of numbers; how many levels each has, from the numbers themselves; and how many cells you are filling, from their product. A third factor adds a third number, so 2 × 2 × 4 is sixteen cells.

Two properties of a design matter for everything below. It is fully crossed when every cell is actually filled, which is what lets one analysis answer three questions instead of one. It is balanced when every cell holds the same number of observations, which is what keeps the arithmetic tidy.

Three questions for the price of one

A two-way design with factors A and B answers three separate questions:

• Main effect of A: does A matter, averaging over B?
• Main effect of B: does B matter, averaging over A?
• A × B interaction: does the effect of A depend on the level of B? This is the one you can only get by studying them together.

The interaction plot

The fastest way to read a two-way result is a picture. Put factor A on the x-axis and draw one line for each level of B. The rule is simple: parallel lines mean no interaction (A's effect is the same at every level of B); non-parallel lines mean there's an interaction. Drag the cell means below and watch it happen. The table beside the plot keeps the same four numbers in the square arrangement a printed report uses, with every margin and every simple effect worked out from them.

Margins, and the difference of differences

Everything the analysis has to say about the pattern is already in those four cell means, and getting it out needs no arithmetic harder than averaging. Take the numbers the widget boots with. The A₁ row holds 60 at B₁ and 40 at B₂; the A₂ row holds 75 and 55.

Average the two cells in a row and the answer is a marginal mean for that level of A: 50 for A₁, 65 for A₂. The gap between those margins, 15 points, is the main effect of A. Average down the columns instead and B's margins are 67.5 and 47.5, a main effect of 20. Average all four cells and you have the grand mean, 57.5.

The word marginal is literal: write the four cells in a square and the row and column averages sit in the margins around it, which is where a printed table puts them and where the Estimated Marginal Means box in SPSS gets its name. One caution travels with that box. A marginal mean is the unweighted average of the cells in its row, so once cells hold different numbers of people it stops matching the plain average of everyone in that row. In a balanced design the two agree exactly.

Now stay inside one column. The simple main effect of A at B₁ is what A does at that one level of B, which with two levels is a difference of two cells: 75 − 60 = 15. At B₂ it is 55 − 40 = 15. Those two numbers are the slopes of the two lines in the plot, and each main effect is nothing more than the average of its own two simple main effects, which is what "averaging over the other factor" means when you write it out.

Subtract one simple main effect from the other and you have the difference of differences: 15 − 15 = 0. A does the same thing at both levels of B, the lines are parallel, and there is no interaction. Press the widget's Interaction button and the same subtraction returns 15 − 35 = −20, which is what non-parallel looks like as a number.

Nothing in this privileges A. The simple main effects of B are the vertical gaps within a column of the plot, 60 − 40 = 20 at A₁ and 75 − 55 = 20 at A₂, and their difference is the same 0. An interaction is symmetric: if A's effect changes across the levels of B then B's effect changes across the levels of A, by the same amount, and only the sign depends on which factor you differenced first. Two effects, one fact.

Reading the patterns

With A on the x-axis and one line per level of B, five arrangements cover almost everything a 2 × 2 can do, and each one names the effects that are present:

Sloping means the row margins differ, so A has a main effect. Separated means the column margins differ, so B has one. Only the last row needs care, because non-parallel lines can sit above perfectly equal margins, and then the main effect is zero while plenty is going on.

Non-parallel splits into two named shapes. A spreading interaction, which this site has been calling fanning, keeps both simple main effects pointing the same way and changes only their size: A helps at both levels of B, more at one than the other. A crossover interaction, the crossing case, reverses the sign, so A helps at one level of B and hurts at the other and the two lines meet. Crossover is also called disordinal, because the ordering of the levels itself flips. The widget's third preset is that case, and it is the one most likely to be missed by a study that measured only A.

Interaction is moderation

The same claim has a second vocabulary, worth knowing because two other parts of the site use it. A variable that changes the strength or the direction of another variable's effect is a moderator, and what it does is moderation. Age moderates the drug's effect; prior expertise moderates the teaching method's. The design tradition says interaction and the modeling tradition says moderation, and interactions in regression shows they are one model, written once with factors and once with continuous predictors.

There are two good reasons to build a study around an interaction rather than meet one by accident. The first is generalization. A main effect established in one population, one setting or one task carries an implied "and everywhere else", and crossing the manipulation with a second factor is how you find the boundary where it stops holding. The second is theory: some accounts predict an interaction directly, and then the interaction is the test rather than a qualification of one. If a memory advantage is supposed to come from effortful retrieval, it should appear when retrieval is effortful and vanish when it is easy, and no main effect can say that.

Often the interaction is the finding. "The drug works" is fine; "the drug works for young patients but not old ones" is far more useful. A famous trap runs the other way: reporting only main effects when a strong interaction is present can be downright misleading. If a crossover makes A helpful for B₁ and harmful for B₂, the average main effect of A might be zero, hiding two large and opposite real effects.

The two-way ANOVA table

The picture says which pattern the means make. The table says whether any of it survives the noise, and completing one is what a factorial exam question asks for. It carries two more source lines than the one-way table, because the variation between cells now splits three ways instead of staying whole.

I is the number of levels of A, J the number of levels of B, and IJ the number of cells. The five degrees of freedom add up, as they always do: (I − 1) + (J − 1) + (I − 1)(J − 1) + (N − IJ) is N − 1. In a 2 × 2 the first three are all 1, which is why a 2 × 2 prints three F ratios with a 1 on top of each.

With n observations in every cell, and writing x̄_i· for a row margin, x̄_·j for a column margin, x̄_ij for a cell mean and x̄ for the grand mean, the sums of squares are:

The bracket inside SS_AB is the difference of differences again, one cell at a time. Take a cell mean, subtract what its row margin and column margin between them lead you to expect, and what is left over is that cell's share of the interaction. In a 2 × 2 those four leftovers are the same size with alternating signs, each one a quarter of the difference of differences.

Three F ratios come out of that table and every one of them divides by the same MS_E. That single error term is the pooled within-cell variance, the two-way version of the pooled spread the follow-up comparisons borrow, and its square root s_p = √MS_E is the number to report as the residual spread. For effect sizes the usual choice is partial eta squared, η_p² = SS_effect / (SS_effect + SS_E), which asks what share of the variation an effect accounts for once the other effects have been set aside. It is what the software prints, and because its denominator changes from row to row, the three values in a table do not add to anything.

All of this assumes a balanced design. When cells hold different numbers of people the four sums of squares stop adding to SS_T, because the effects are no longer separable, and software has to be told which of several conventions to use for splitting the overlap. The default nearly everywhere is called Type III, which asks what each effect adds once every other effect is already in the model; it is the right default, and it is worth knowing the choice exists before an output table surprises you.

The study it boots with crosses practice time (short or long) with feedback (none or coached) and scores a skills test out of 40, four people in each cell. Its cell means are 10 and 14 in the short-practice row, 16 and 28 in the long-practice row, so SS_A = 400, SS_B = 256, SS_AB = 64 and SS_E = 60 against a total of 780. The two-way table gives MS_E = 5.00 on 12 degrees of freedom and F(1, 12) = 80.00 for practice time. Analyzed as a one-way ANOVA on practice time alone, the same 400 is tested against an error of 380 on 14 degrees of freedom, so MS_E = 27.14 and F(1, 14) = 14.74. Same data, same effect, and one analysis is more than five times as sharp about it.

Push the feedback slider to its maximum and the one-way F falls to 3.25, which on 1 and 14 degrees of freedom does not reach the 4.60 it would need at α = .05, while the two-way F for practice time has not moved off 80.00 the whole way. A real effect that a well-designed study finds easily can be invisible to a study that ignored the other thing in the room.

Two factors are cheaper than two studies

That the second factor sharpens the first test is one argument for factorial designs. The other is arithmetic about people. Consider a 3 × 3 with 14 participants in each of its nine cells, which is 126 people. Every level of A is then represented by 42 of them, and so is every level of B, because each person sits in one level of each. To get 42 per group from separate studies you would run a three-group study of A with 126 people and a three-group study of B with another 126, and the 252 you spent would still tell you nothing about the interaction. The factorial design asks both questions, and a third one neither of the separate studies could ask, for half the sample.

None of this is free. Each extra source line takes degrees of freedom away from the error term, and in a small study those can cost more than the variation they remove; the widget above goes from 14 error df to 12 to buy its 320 points of named variation, which is a bargain, but it would not be if the sums of squares were tiny and the df were scarce. The usual case is the widget's case, and it is why a factor worth crossing in is worth crossing in even when its own main effect is not the question you came to ask.

Interpret the interaction first

The reading order for a two-way table is fixed and it is not the order the table prints in. Look at the A × B line first. If it is significant, the two main effects above it describe averages that may not describe anybody, and the honest write-up leads with the interaction and then reports each main effect as qualified by it, usually alongside the simple main effects that say what happened at each level. If the interaction is not significant, the main effects mean what they say and you can report them plainly.

One misreading is common enough to name. A main effect that comes out nonsignificant under a significant interaction does not say the factor never matters. It says the factor's effects at the different levels of the other factor averaged out, which is a statement about an average and not about either level. In a full crossover the main effect can be exactly zero while both simple main effects are large and both are real. Test the simple main effects before concluding that a factor did nothing.

Problem 15 of the practice problems reads a caffeine × sleep interaction off a 2 × 2 table of cell means, Problem 30 works a full set of margins and simple main effects out of four cell means, and Problem 31 completes a two-way table with four cells blanked. Problem 26 is the trap above: an abstract that reports the main effect when the interaction is the story. To watch one 2 × 2 study travel from its hypothesis all the way to its APA paragraph, follow the complete worked project.