Difference-in-Differences
A policy arrived in some places and not others, and you have records from before and after. The whole answer is a single coefficient on the product of two dummy variables. Quasi-experiments in the Research Toolkit explains why the method works. Comparing the groups afterward is biased by whatever gap they started with. Comparing the treated group's own before and after is biased by whatever was changing anyway. Taking the difference of the two differences cancels both. This lesson continues from there, with the regression that does the subtraction, the standard errors it needs, and the assumption it depends on.
One coefficient, four cells
Start with the smallest version. A regional education authority introduced a peer-tutoring scheme in 25 of its 50 schools from the fifth term of an eight-term record. The outcome is each school's mean end-of-term score. Average the terms before the switch and the terms after it, for each group, and you have four numbers:
| Group | Before | After | Change |
|---|---|---|---|
| Tutoring schools (25) | 64.1 | 70.6 | +6.5 |
| Comparison schools (25) | 61.9 | 65.8 | +3.9 |
| Difference | 2.2 | 4.8 | 2.6 |
The bottom-right cell is the estimate, and you can read it in two ways that give the same number. The tutoring schools gained 6.5 and the comparison schools gained 3.9, so the tutoring schools gained 2.6 points more. Or the gap between the groups widened from 2.2 to 4.8, an increase of 2.6. Those are the two differences the design is named after, and it does not matter which one you take first.
The regression that does it
Stack the data so that every school contributes one row per period, and build two indicators. Treated is 1 for a school in the scheme and 0 otherwise, in every period, including the ones before the scheme existed. Post is 1 in the periods after the switch and 0 before it, for every school, including the ones that never joined. Then fit
Y = b0 + b1Treated + b2Post + b3(Treated × Post) + e
and the four coefficients are the four cells of that table, rearranged. b0 is the comparison group before the switch, 61.9. b1 is the baseline gap between the groups, 2.2. b2 is the comparison group's own change over time, 3.9. And b3, the coefficient on the interaction, is 2.6, the estimate. Add all four and you get 70.6, the treated group's mean after the switch. So the four coefficients reproduce all four cell means.
Two of those coefficients are often misread. b1 is not an effect of the scheme. It is the difference between the two groups of schools before any scheme existed, and the design is built to remove it. b2 measures whatever else happened to every school between the two periods, and the design subtracts that too. Only b3 estimates the effect. A results table that reports all four with significance stars invites readers to interpret three numbers that say nothing about the scheme.
The same model, written the way papers write it
With two groups and two periods the specification above is complete. Real data usually have more of both, and then the standard form replaces each indicator with a full set of dummies:
Yit = ai + gt + bDit + eit
where ai is a dummy for each unit, gt is a dummy for each period, and Dit is 1 when unit i is being treated at time t. This is two-way fixed effects. The unit dummies absorb anything about a school that never changes, and the period dummies absorb anything that affected every school at once. It is the same estimator. With two groups and two periods, a dummy per unit and a dummy per period span exactly the same space as Treated, Post and an intercept, so b and b3 are the identical number. Almost every paper you read will use the fixed-effects form, and it handles fifty schools as easily as two.
What changes with the longer panel is the standard error. A school observed in eight terms contributes eight rows, and those rows are not eight independent observations: a school that is unusually good is unusually good in all of them. Ordinary least squares treats every row as independent and returns standard errors that are far too small, sometimes by a factor of two or three. The standard fix is clustered standard errors, grouped at the level where the treatment was assigned, here the school, so that each school counts once and not eight times. In this simple case you can compute it exactly by hand. Reduce each school to a single number, its own after-minus-before change, and compare the 25 changes in one group with the 25 in the other using an independent-samples t-test. For the tutoring data that gives 2.60 points, SE = 0.63, t(48) = 4.13, p < .001, 95% CI [1.33, 3.87].
🎮 Trends, Parallel and Otherwise
Fifty schools across eight terms. Twenty-five start the tutoring scheme in term 5 (indigo), twenty-five never do (teal). The 2 × 2 view averages the four cells the estimate is built from. The event study compares the same schools one term at a time against term 4, so the three coefficients before the switch show whether the groups were already drifting apart before the scheme began.
Parallel trends, and why no test reaches it
The subtraction works only if the comparison group's change is a good stand-in for what the treated group would have done without treatment. That is the parallel-trends assumption. It allows the two groups to be at different levels throughout, so b1 can be large. It does not allow them to change at different rates. And it is a statement about an outcome no one observed, the treated group's outcome in a world where the scheme never arrived, so no dataset has a column to test it against. Any evidence a paper offers for it is indirect.
The best indirect evidence comes from the periods before the treatment. Both groups were untreated then, so if they diverged, the design has no explanation for it. The formal version is an event study, which replaces the single before-and-after contrast with a separate coefficient for each period, measured against a reference period just before the switch. In the tutoring data, with term 4 as the reference, that means seven coefficients for terms 1, 2, 3, 5, 6, 7 and 8. Each is the gap between the groups in that term minus the gap in term 4. With only two groups you can compute them without any regression, as the widget above does.
Three of those coefficients are for terms before the scheme existed, and under parallel trends they should be noise around zero. Suppose instead that they come back at −1.8, −1.2 and −0.6, rising steadily toward the reference. That is a differential trend of 0.6 points a term, visible three terms early, and there is no reason for it to stop at term 4 just because the scheme started then. Counted from term 5, the average term before the switch is 2.5 terms back and the average term after it is 1.5 terms forward. They are four terms apart, so the drift puts 4 × 0.6 = 2.4 points into the estimate. Of the 2.60 the table reported, only 0.2 would be left. The 2 × 2 table cannot show you this, because averaging each period into one number hides the trend.
A flat pre-trend does not prove that the trends are parallel. A pre-trend test can pass because the trends really are parallel, because the pre-period is short, or because the outcome is so noisy that a real drift cannot be distinguished from zero. Set a differential trend in the widget and then turn the noise slider up. The coefficients before term 5 widen until their intervals cover zero easily, while the estimate stays exactly as wrong as it was. Three pre-periods with wide intervals are not a real test of the assumption. Say how many pre-periods there were and how precisely each was estimated, so readers can judge for themselves.
Some violations leave the pre-periods untouched. Suppose the authority had also replaced the principals of the 25 tutoring schools in term 5. The event study would look just as clean, and the 2.6 would mix the effect of tutoring with the effect of new principals. A shock that starts on the same date in the treated units alone cannot show up before that date. Only knowledge of what else changed can rule it out, and a paper should say what it checked.
Two further checks give the assumption a chance to fail. A placebo date pretends the scheme began earlier, say in term 3, and reruns the analysis on terms 1 to 4 alone. It should find nothing. In the drifting example above it would find (−0.6 + 0) / 2 − (−1.8 − 1.2) / 2 = 1.2 points, the effect of a scheme that did not exist yet. With a single start date this asks the same question as the event study's pre-period coefficients, in one number. A second comparison group, such as the schools of a neighboring authority, should give a similar estimate. If two plausible comparison groups give clearly different answers, they cannot both be good stand-ins for the tutoring schools.
Many periods, and treatments that start at different times
Two extensions come up all the time. The first is easy: more than two periods, with everyone treated at the same moment. The two-way fixed-effects regression handles that unchanged, and the event study becomes the natural way to report it. The only new question is whether the effect grows, fades or holds steady across the post-treatment periods, and the event study answers it by showing each period separately.
The second is not easy. Under staggered adoption, units start the treatment at different dates: a few districts in 2015, more in 2017, the rest in 2019. For two decades the standard practice was to run the same two-way fixed-effects regression, and it can be badly wrong. With staggered timing, the regression averages many small comparisons without reporting them, and some of them use already-treated units as the comparison group for later-treated ones. When the effect changes over time, those comparisons subtract one group's treatment effect from another's. They can enter the average with negative weight, and that can flip the sign of the estimate even when every underlying effect is positive.
A decomposition published by Goodman-Bacon in 2021 made that arithmetic explicit. It writes the two-way estimate as a weighted average of every two-group, two-period comparison in the data and shows each comparison's weight, and reporting it is now standard practice. Newer estimators avoid the problem comparisons by design. The Callaway and Sant'Anna estimator computes a separate effect for each adoption cohort at each period, using units not yet treated as the comparison. The Sun and Abraham interaction-weighted estimator makes the same correction inside an event study. Both are one command in R or Python. Learn their names mainly so you can spot the problem in other people's work: a paper that runs plain two-way fixed effects on staggered timing without addressing the issue has a question to answer.
What to report
The coefficient is the easy part of a difference-in-differences result. To make it interpretable, say which units were treated and when, and what the comparison group is and why it is plausible. Then say how the standard errors were clustered, how many pre-periods were available, and what the event study showed in them. If some schools drop out of the record partway, say how many and when. A panel that loses units after the switch has a missing-data problem, and the regression runs without warning on whoever is left. Like the instrument and the cutoff before it, this design is only as credible as its assumption, and readers are entitled to see that assumption defended. Reading a Causal-Inference Paper is the guide for when you are the one reading, and it includes a printable checklist for the three designs.
Why it matters: policies often start in one place before another, and each difference in timing is a comparison someone could analyze. Most published evidence about minimum wages, school reforms, health insurance and traffic laws comes from difference-in-differences, so it is the design you are most likely to have to evaluate. Its arithmetic takes ten minutes to learn. Judging whether the comparison group was a good stand-in takes the rest of this lesson, and sometimes a whole seminar. Section 3.12 gives you the vocabulary for that argument.
Common questions
Do the same units have to appear in every period?
Two different data shapes both support the design, and which one you have decides how to compute the standard errors. A panel follows the same units over time, so school 14 appears in all eight terms. Cluster on the unit, because those rows are repeated measurements of the same school. Repeated cross-sections survey new people each period from the same populations, so no one appears twice, and the design compares group averages, not individual changes. Clustering then belongs at the level where the treatment varies, usually the region or the school and not the respondent. The estimator is identical in both cases. What differs is the meaning of a change in who was sampled. If the kind of person sampled in the treated areas changed between periods, that change is part of your estimate and clustering cannot remove it, so report the composition of each cell alongside the result.
How many pre-treatment periods do I need?
One is enough to compute the estimate and nowhere near enough to defend it. With a single period before the treatment there is nothing to compare against, so parallel trends is simply asserted. The design still runs, but a careful reader will trust it much less. Three or four pre-periods let an event study show whether the groups were moving together, and that is the minimum most reviewers now expect. More is better, with one caveat. A long pre-period gives a pre-trend test more power, so it detects small drifts that a short one would miss. A design that looks clean with two pre-periods can look doubtful with eight even though nothing about the world has changed, and that means the test is doing its job. If the pre-period is short, say so in the sentence where you state the assumption, so readers do not have to count the columns in your figure.
If I take logs of the outcome, does parallel trends still hold?
Not automatically. The assumption depends on the scale the outcome is measured on. Parallel trends in levels says the two groups would have changed by the same number of units. Parallel trends in logs says they would have changed by the same percentage. Those are different claims, and they cannot both be true unless the groups started at the same level. A comparison group at 40 and a treated group at 80, both growing 10% a year, are parallel in logs and diverging by four units a year in levels. The level specification would then report a treatment effect produced entirely by that arithmetic. No test settles which scale is right. Pick the scale on which the outcome is naturally generated, say why in a sentence, and show the result on the other scale as a robustness check. A finding that appears on only one scale is probably a product of the transformation, not of the treatment.