Between vs Within Designs
Once you've decided what to compare, you still have to decide who experiences each condition. In a between-subjects design, different people get different conditions. In a within-subjects (repeated-measures) design, the same people get all of them. That one choice can multiply or divide the number of participants you need, and it comes with a catch.
Why within-subjects can be so efficient
The enemy of a between-subjects comparison is individual differences. If two groups differ, is it the treatment, or just that these people were always faster, smarter, calmer? All that person-to-person noise sits in the denominator of your test statistic, drowning the signal. A within-subjects design lets each person serve as their own control: you look at how each individual changed, so their stable quirks subtract out. When people are consistent (their scores in the two conditions correlate strongly), that cancellation is enormous — which is exactly why the paired t-test and, for three conditions or more, repeated-measures ANOVA are so powerful.
⚖️ The Same Study, Two Designs
One true effect, run as a between-subjects and a within-subjects study. The curves show statistical power against the number of participants; the readouts give the sample each design needs for 80% power. Turn up individual differences to reward the within design, then turn up carryover and watch its edge collapse.
Notice the crossover: the within design only wins when people's two scores correlate above about 0.5. Below that, splitting the difference between two noisy measurements actually costs you, and a plain between-subjects design is the safer bet.
The catch: order and carryover effects
Measuring the same person twice creates problems a between design never has:
- Practice / fatigue: people get better (or more tired) simply from doing the task again, regardless of condition.
- Carryover: the first condition lingers into the second — a drug still in the bloodstream, a strategy just learned, a mood just induced.
- Demand characteristics: seeing both conditions, participants may guess the hypothesis and play along.
Counterbalancing (and its limits)
The standard defense is counterbalancing: give half the participants condition A first and half condition B first, so any pure order effect cancels out on average. With more than two conditions you can't run every possible order, so researchers use a Latin square (a schedule in which each condition appears once in every position) to spread order effects evenly without needing all k! sequences.
But counterbalancing only tames symmetric order effects. If A changes B more than B changes A (asymmetric transfer), balancing the orders leaves residual noise in the difference scores, which is what the carryover slider above is doing to the within design's power. Sometimes within-subjects is simply impossible: you can't un-teach a lesson, un-reveal a deception, or randomly reassign someone's gender. Then between-subjects is your only option.
The trade-off in one line: within-subjects buys power by removing individual differences, and pays for it with order and carryover effects. Reach for it when conditions are reversible and people are consistent; fall back to between-subjects when they aren't, and counterbalance whenever you do go within.
Why it matters: the design decision happens before you collect a single data point, and it's largely irreversible. Choosing within-subjects for a strongly-correlated, reversible manipulation can cut your required sample by half or more; choosing it for a task with heavy carryover can wreck the study. Match the design to the manipulation, not to convenience.
Problem 27 of the practice problems puts the choice in front of you with a budget attached: 90 sessions to spend on three interface designs, large individual differences in typing speed, and a decision to defend either way.
Common questions
What is a mixed design?
A design with at least one between-subjects factor and at least one within-subjects factor in the same study, and it is extremely common once you look for it. Any experiment with a treatment group and a control group (between) measured before and after (within) is a mixed design, as is a training study where two teaching methods are compared across three timepoints. You get the within factor's efficiency where you can have it and accept the between factor's cost where the manipulation cannot be reversed. The analysis is a repeated-measures ANOVA with the between factor added, and the effect people usually care about is the interaction: whether the two groups changed differently over time, which is a sharper question than whether either changed at all.
Are within-subjects designs always more powerful?
No. They win only when a person's scores across conditions correlate strongly — roughly above 0.5. That correlation is what lets individual differences cancel out; below it, you'd have been better off with independent groups. And heavy carryover (practice, fatigue, a lingering manipulation) adds noise that only the within design pays, which can erase the advantage entirely. When conditions correlate strongly and carry over little, though, a within design can need less than half the participants for the same power.
Does counterbalancing remove carryover effects?
Only partly. Counterbalancing (running the conditions in different orders across participants, e.g. via a Latin square) averages out symmetric order effects so they don't bias the mean. But if the transfer is asymmetric (condition A changes B more than B changes A), balancing the orders leaves residual variability in the difference scores rather than removing it. Counterbalancing is a defense, not a cure; when carryover is severe, a between-subjects design is safer.