Factorial ANOVA (Two-Way)

Real situations rarely involve just one factor. Does a drug's effect depend on dose and on age? Does a teaching method work differently for beginners than experts? A factorial ANOVA tests two (or more) factors at once. It can also test something no single-factor study can: the interaction, whether the effect of one factor depends on the other.

Factors, levels, and cells

Three words cover most of the notation. A factor is a categorical variable the study manipulates or classifies people by. Its levels are the categories it takes, so a factor called Dose might have the levels None, Low and High. Cross two factors, pairing every level of one with every level of the other, and each pairing is a cell.

The design is then named by its levels, multiplied. Two levels of each factor is a 2 × 2 with four cells. Three levels of each is a 3 × 3 with nine. Four doses crossed with two age bands is a 4 × 2 with eight. The notation tells you three things: how many numbers there are is the number of factors, each number is that factor's number of levels, and their product is the number of cells. A third factor adds a third number, so 2 × 2 × 4 is sixteen cells.

Two properties of a design matter for everything below. It is fully crossed when every cell has observations in it, and that allows one analysis to answer three questions. It is balanced when every cell holds the same number of observations, and that keeps the arithmetic simple.

Three questions for the price of one

A two-way design with factors A and B answers three separate questions:

• Main effect of A: does A matter, averaging over B?
• Main effect of B: does B matter, averaging over A?
• A × B interaction: does the effect of A depend on the level of B? You can only test this by studying the two factors together.

The interaction plot

The fastest way to read a two-way result is a picture. Put factor A on the x-axis and draw one line for each level of B. Parallel lines mean no interaction (A's effect is the same at every level of B), and non-parallel lines mean there is an interaction. Drag the cell-mean sliders below to see this. The table beside the plot shows the same four numbers in the square layout a printed report uses, with every margin and every simple effect worked out from them.

Margins, and the difference of differences

Everything the analysis says about the pattern is already in those four cell means, and you need nothing harder than averaging to get it out. Take the widget's starting numbers. The A₁ row has 60 at B₁ and 40 at B₂, and the A₂ row has 75 and 55.

Average the two cells in a row and the answer is a marginal mean for that level of A: 50 for A₁, 65 for A₂. The gap between those margins, 15 points, is the main effect of A. Average down the columns instead and B's margins are 67.5 and 47.5, a main effect of 20. Average all four cells and you have the grand mean, 57.5.

The word marginal is literal. Write the four cells in a square and the row and column averages go in the margins around it. A printed table puts them there, and the Estimated Marginal Means box in SPSS is named after them. One caution about that box. A marginal mean is the unweighted average of the cells in its row. Once cells hold different numbers of people, it no longer matches the plain average of everyone in that row. In a balanced design the two agree exactly.

Now stay inside one column. The simple main effect of A at B₁ is the effect of A at that one level of B. With two levels of A, it is the difference of two cells: 75 − 60 = 15. At B₂ it is 55 − 40 = 15. Those two numbers are the slopes of the two lines in the plot. Each main effect is the average of its own two simple main effects, and that is what "averaging over the other factor" means.

Subtract one simple main effect from the other and you have the difference of differences: 15 − 15 = 0. A does the same thing at both levels of B, the lines are parallel, and there is no interaction. Press the widget's Interaction button and the same subtraction gives 15 − 35 = −20. Non-parallel lines give a difference of differences that is not zero.

The same works with the factors swapped. The simple main effects of B are the vertical gaps between the lines at each level of A: 60 − 40 = 20 at A₁ and 75 − 55 = 20 at A₂. Their difference is the same 0. An interaction is symmetric. If A's effect changes across the levels of B, then B's effect changes across the levels of A by the same amount. Only the sign depends on which factor you subtracted first.

Reading the patterns

With A on the x-axis and one line per level of B, five arrangements cover almost every 2 × 2 result. The table lists the effects each one shows:

Sloping lines mean the row margins differ, so A has a main effect. Separated lines mean the column margins differ, so B has one. Only the last row needs care. Non-parallel lines can come with exactly equal margins, and then the main effect is zero even though the simple main effects are not.

Non-parallel splits into two named shapes. A spreading interaction (also called fanning) keeps both simple main effects in the same direction and changes only their size: A helps at both levels of B, but more at one than the other. A crossover interaction reverses the sign: A helps at one level of B and hurts at the other, and the two lines meet. Crossover is also called disordinal, because the ordering of the levels itself flips. The widget's third preset is a crossover, and it is the pattern most likely to be missed by a study that measured only A.

Interaction is moderation

Other lessons describe the same idea with different words. A variable that changes the strength or the direction of another variable's effect is a moderator, and what it does is moderation. Age moderates the drug's effect; prior expertise moderates the teaching method's. The design tradition says interaction and the modeling tradition says moderation, and interactions in regression shows they are one model, written once with factors and once with continuous predictors.

There are two good reasons to design a study around an interaction. The first is generalization. A main effect found in one population, one setting or one task is usually assumed to hold everywhere else. Crossing the manipulation with a second factor shows where it stops holding. The second is theory. Some theories predict an interaction directly, and then the interaction is the main test, not a qualification of one. If a memory advantage comes from effortful retrieval, it should appear when retrieval is effortful and vanish when it is easy, and no main effect can test that.

Often the interaction is the finding. "The drug works" is useful, but "the drug works for young patients but not old ones" is far more useful. The opposite mistake is common: reporting only main effects when a strong interaction is present can be misleading. If a crossover makes A helpful for B₁ and harmful for B₂, the average main effect of A might be zero, hiding two large and opposite real effects.

The two-way ANOVA table

The plot shows the pattern the means make. The table tells you which parts of that pattern are statistically significant, and factorial exam questions usually ask you to complete one. It has two more source lines than the one-way table, because the variation between cells now splits three ways.

I is the number of levels of A, J the number of levels of B, and IJ the number of cells. The degrees of freedom add up, as they always do: (I − 1) + (J − 1) + (I − 1)(J − 1) + (N − IJ) is N − 1. In a 2 × 2 the first three are all 1, so a 2 × 2 prints three F ratios, each with 1 as its first df.

With n observations in every cell, and writing x̄_i· for a row margin, x̄_·j for a column margin, x̄_ij for a cell mean and x̄ for the grand mean, the sums of squares are:

The bracket inside SS_AB is the difference of differences again, one cell at a time. Take a cell mean and subtract what its row margin and column margin together predict. The remainder is that cell's share of the interaction. In a 2 × 2 those four leftovers are the same size with alternating signs, each one a quarter of the difference of differences.

The table gives three F ratios, and all three divide by the same MS_E. That error term is the pooled within-cell variance, the two-way version of the pooled spread the follow-up comparisons use. Its square root, s_p = √MS_E, is the number to report as the residual spread. For effect sizes the usual choice is partial eta squared, η_p² = SS_effect / (SS_effect + SS_E): the share of the variation an effect accounts for once the other effects have been set aside. Software prints it, and because its denominator changes from row to row, the three values in a table do not sum to anything meaningful.

All of this assumes a balanced design. When cells hold different numbers of people, the four sums of squares no longer add to SS_T, because the effects are no longer separable. Software then has to use one of several conventions for splitting the overlap. The default nearly everywhere is Type III, which tests what each effect adds once every other effect is already in the model. It is the right default, but know that the choice exists, because different settings print different tables from the same data.

The starting study crosses practice time (short or long) with feedback (none or coached). The outcome is a skills test scored out of 40, with four people in each cell. Its cell means are 10 and 14 in the short-practice row and 16 and 28 in the long-practice row, so SS_A = 400, SS_B = 256, SS_AB = 64 and SS_E = 60, out of a total of 780. The two-way table gives MS_E = 5.00 on 12 degrees of freedom and F(1, 12) = 80.00 for practice time. Analyzed as a one-way ANOVA on practice time alone, the same 400 is tested against an error of 380 on 14 degrees of freedom, so MS_E = 27.14 and F(1, 14) = 14.74. The data and the effect are the same, but the two-way F is more than five times as large.

Push the feedback slider to its maximum and the one-way F falls to 3.25. On 1 and 14 degrees of freedom it would need 4.60 to be significant at α = .05. The two-way F for practice time stays at 80.00 the whole way. So a real effect that a well-designed study finds easily can be missed by a study that leaves out another factor affecting the outcome.

Two factors are cheaper than two studies

A more sensitive test of the first factor is one argument for factorial designs. The other is about sample size. Consider a 3 × 3 with 14 participants in each of its nine cells, 126 people in all. Every level of A then has 42 of them, and so does every level of B, because each person is in one level of each. To get 42 per group from separate studies, you would need a three-group study of A with 126 people and a three-group study of B with another 126. Those 252 people would still tell you nothing about the interaction. The factorial design answers both questions, plus a third that neither separate study could answer, with half the sample.

Adding a factor has a cost. Each extra source line takes degrees of freedom away from the error term, and in a small study that loss can outweigh the variation the new lines remove. The widget above gives up 2 error df (14 down to 12) to move 320 points of variation out of the error, which is clearly worth it. It would not be if those sums of squares were tiny and the df were scarce. Usually the trade is worth making, so a factor that affects the outcome is worth crossing in even when its own main effect is not your research question.

Interpret the interaction first

Read a two-way table in a fixed order, which is not the order it prints in. Look at the A × B line first. If it is significant, the two main effects above it are averages that may not describe any real condition. The write-up should lead with the interaction and then report each main effect as qualified by it, usually alongside the simple main effects that say what happened at each level. If the interaction is not significant, the main effects can be read and reported as they stand.

One misreading is common. A nonsignificant main effect alongside a significant interaction does not mean the factor never matters. It means the factor's effects at the different levels of the other factor averaged out. That is a statement about the average, not about either level. In a full crossover the main effect can be exactly zero while both simple main effects are large and both are real. Test the simple main effects before concluding that a factor did nothing.

Problem 36 of the practice problems reads a caffeine × sleep interaction off a 2 × 2 table of cell means. Problem 51 works out a full set of margins and simple main effects from four cell means, and Problem 52 completes a two-way table with four cells blanked. Problem 66 is the mistake described above: an abstract that reports the main effect when the interaction is the main finding. To follow one 2 × 2 study from its hypothesis to its APA paragraph, see the complete worked project.