Between vs Within Designs
Once you've decided what to compare, you still have to decide who experiences each condition. In a between-subjects design, different people get different conditions. In a within-subjects (repeated-measures) design, the same people get all of them. That one choice can multiply or divide the number of participants you need, and it comes with a catch.
Why within-subjects can be so efficient
The main problem for a between-subjects comparison is individual differences. If two groups differ, is it the treatment, or just that these people were always faster, smarter, calmer? All that person-to-person variation goes into the denominator of your test statistic, where it makes the effect harder to detect. A within-subjects design lets each person serve as their own control: you look at how each individual changed, so their stable quirks subtract out. When people are consistent (their scores in the two conditions correlate strongly), most of that variation cancels. The paired t-test and, for three conditions or more, repeated-measures ANOVA are powerful for this reason.
⚖️ The Same Study, Two Designs
One true effect, run as a between-subjects and a within-subjects study. The curves show statistical power against the number of participants; the readouts give the sample each design needs for 80% power. Turn up individual differences and the within design needs fewer people. Then turn up carryover and watch that advantage disappear.
Watch the dz readout. It is larger than d only when people's two scores correlate above about 0.5. Below that, the within design still needs fewer people, but only because it measures each person twice, and a little carryover is enough to make a plain between-subjects design the better choice.
The catch: order and carryover effects
Measuring the same person twice creates problems a between design never has:
- Practice / fatigue: people get better (or more tired) simply from doing the task again, regardless of condition.
- Carryover: the first condition lingers into the second: a drug still in the bloodstream, a strategy just learned, a mood just induced.
- Demand characteristics: seeing both conditions, participants may guess the hypothesis and play along.
Counterbalancing (and its limits)
The standard defense is counterbalancing: give half the participants condition A first and half condition B first, so any pure order effect cancels out on average. With more than two conditions you can't run every possible order. Researchers then use a Latin square, a schedule in which each condition appears once in every position, to spread order effects evenly without needing all k! sequences.
But counterbalancing only removes symmetric order effects. If A changes B more than B changes A (asymmetric transfer), balancing the orders leaves residual noise in the difference scores. The carryover slider above adds that noise to the within design. Sometimes within-subjects is simply impossible: you can't un-teach a lesson, un-reveal a deception, or randomly reassign someone's gender. Then between-subjects is your only option.
The trade-off: within-subjects gains power by removing individual differences, at the cost of order and carryover effects. Use it when conditions are reversible and people are consistent, and counterbalance the orders when you do. Otherwise, use between-subjects.
Why it matters: the design decision happens before you collect a single data point, and it's largely irreversible. Choosing within-subjects for a strongly correlated, reversible manipulation can cut your required sample by half or more. Choosing it for a task with heavy carryover can ruin the study.
Problem 67 of the practice problems gives you the choice with a budget: 90 sessions for three interface designs, and large individual differences in typing speed. You pick the design and defend it.
Common questions
What is a mixed design?
A design with at least one between-subjects factor and at least one within-subjects factor in the same study. It is very common. Any experiment with a treatment group and a control group (between) measured before and after (within) is a mixed design, as is a training study where two teaching methods are compared across three timepoints. You get the within factor's efficiency where you can have it and accept the between factor's cost where the manipulation cannot be reversed. The analysis is a repeated-measures ANOVA with the between factor added. The effect people usually care about is the interaction: whether the two groups changed differently over time. That is a sharper question than whether either changed at all.
Are within-subjects designs always more powerful?
No. The advantage depends on how strongly a person's scores across conditions correlate, because the correlation sets how much of the individual differences cancels out. Near zero, nothing cancels: the within design needs fewer people only because it measures each person twice, and it gains almost nothing per measurement. Its effect size dz exceeds the between-subjects d only above a correlation of about 0.5. Carryover (practice, fatigue, a lingering manipulation) adds noise that only the within design suffers, and it can erase the advantage entirely. When conditions correlate strongly and carry over little, though, a within design can need less than half the participants for the same power.
Does counterbalancing remove carryover effects?
Only partly. Counterbalancing (running the conditions in different orders across participants, e.g. via a Latin square) averages out symmetric order effects so they don't bias the mean. But if the transfer is asymmetric (condition A changes B more than B changes A), balancing the orders leaves residual variability in the difference scores rather than removing it. So counterbalancing reduces the problem without solving it. When carryover is severe, a between-subjects design is safer.