Instrumental Variables & 2SLS

The previous lesson ends with a rule: adjust for the common causes. That works only for common causes you have measured. Motivation, ability, health at baseline, how much a family already valued the thing you are studying: none of those are in your file. No adjustment can close a backdoor path through a variable you do not have. An instrumental variable gets around the problem. You stop trying to block the confounded comparison and use a source of variation in the treatment that the confounders had no part in.

When the regressor is part of the problem

A regressor is endogenous when it is correlated with the error term, the part of the outcome the model does not explain. Ordinary least squares assumes it is not. When that assumption fails, the coefficient no longer estimates the causal effect. It estimates the effect plus a bias from whatever in the error is correlated with the regressor.

There are three ways this happens, and you should be able to say which one applies to your data:

• Omitted variables. Something causes both the regressor and the outcome and is missing from the model. Section 2.9 derives the size of the bias. The slope you get is the true slope plus the omitted variable's effect on the outcome times the slope from regressing the omitted variable on your predictor. Reread it before going further. Instrumental variables are for the case where the omitted variable cannot be measured and added to the model.
• Reverse causality. Police numbers predict crime rates, and crime rates also determine police numbers. Whichever direction you write the regression in, the effect running the other way ends up in the error.
• Measurement error in the regressor. A predictor recorded with noise is correlated with its own error, and the resulting slope is pulled toward zero. This one surprises people. Random error in measuring the outcome only makes the estimate less precise, while random error in measuring the predictor biases it.

In all three cases, the variation in the regressor that you use to estimate the effect is correlated with the error, and adjustment cannot fix that.

Three conditions, and only one of them is testable

An instrument is a variable, written Z, that satisfies three conditions with respect to your endogenous regressor X and your outcome Y. The course names them in this order and so will we.

• Relevance. Z actually moves X. If the instrument barely shifts the regressor, there is nothing to work with.
• Independence. Z is as good as randomly assigned, so nothing that causes Y also causes Z.
• Exclusion. Z affects Y only by way of X, with no direct path and no path through anything else.

Relevance is the only one your data can test. It is a claim about the association between two variables you observe, and the first stage below measures it. Independence and exclusion are claims about paths that involve the error term, and the error term is by definition unobserved. You support them with an argument about how the instrument came to exist, and a reader either accepts the argument or does not.

First stage, reduced form, and the ratio between them

Two regressions do all the work. Both are ordinary least squares regressions, and that is safe here because in both of them the predictor is the instrument, which is unrelated to the error.

• The first stage regresses the endogenous regressor on the instrument: X = a₀ + a₁Z. Its slope a₁ measures how much the instrument changes X.
• The reduced form regresses the outcome on the instrument: Y = r₀ + r₁Z. Its slope r₁ measures how much the instrument changes Y.

Now use the exclusion restriction. If Z affects Y only through X, then all of r₁ comes through X: the instrument changed X by a₁, and each unit of X changes Y by the causal effect we are after. So r₁ = β × a₁, and dividing gives

β = r₁ / a₁ = (effect of Z on Y) / (effect of Z on X)

which is the Wald ratio. The Greek letter is β (beta), the same causal slope that ordinary regression estimates with bias when X is endogenous.

In the library study, suppose lottery winners attended 12.0 hours on average and losers 4.0 hours, so the first stage is 8.0 hours. Winners scored 68.4 and losers 62.0, so the reduced form is 6.4 points. The Wald ratio is 6.4 / 8.0 = 0.80 points per hour of tutoring. The ordinary regression of score on hours, run on the same 500 people, reports 1.35. The gap of 0.55 points per hour is the omitted-variable term from Section 2.9. Motivation raises both attendance and scores, so it adds 0.55 to the naive slope, and the ordinary regression reports that as an effect of tutoring.

In a randomized study the reduced form has a name of its own. Comparing people by the group the lottery put them in, whatever they did next, is an intention-to-treat analysis, and its 6.4 points is the effect of being offered a place. The randomization protects that comparison. Comparing only the winners who attended with the losers who stayed away, or regrouping people by what they actually did, brings back the motivation gap the lottery removed. The Wald ratio divides the protected comparison by the first stage, turning the effect of an offer into an effect per hour attended.

Two-stage least squares

Two-stage least squares (2SLS) reaches the same answer by a different route, and it extends to bigger models. Fit the first stage and keep its predicted values, X̂. These are the part of the regressor that the instrument explains, so they are uncorrelated with the error. Then regress Y on X̂. The slope of that second regression is the causal estimate.

With one instrument and one endogenous regressor the two routes agree to the last decimal, because X̂ is a linear function of Z and regressing on it recovers exactly r₁/a₁. You still need 2SLS for the cases the Wald ratio cannot handle: several instruments at once, control variables alongside the endogenous regressor, or more than one endogenous regressor.

One warning about running the two stages yourself. If you fit the first stage, save the predictions into a new column and run the second regression by hand, the coefficient is right and the standard errors are wrong. The second regression treats X̂ as observed data, so it computes its residuals from the fitted regressor when they should come from the real one, and it reports more precision than you have. Dedicated 2SLS commands correct this, and the software section below says what to do in a package that has none.

A weak instrument is worse than it looks

The Wald ratio divides by the first stage, so with an instrument that barely changes the regressor you are dividing by something close to zero. Two things go wrong at once. The estimate becomes very variable, because small sampling fluctuations in a small denominator produce large swings in the ratio. It also becomes biased toward the ordinary least squares estimate you were trying to correct. So a weak instrument does not simply fail to give an answer: it tends to give the wrong answer you were trying to avoid.

The usual check is the F statistic from the first stage, which tests whether the instruments predict the regressor at all. The working threshold is about 10. Drag the strength slider down in the widget above, and the standard error and the F change together. The standard error grows by roughly the factor the first stage shrank by, and the F drops below 10 along the way. Treat 10 as a rule of thumb. It is a convention that came from simulations, later work has argued for a higher bar, and a first-stage F of 11 is not meaningfully different from one of 9.

When the exclusion restriction fails

Suppose winning the lottery adds 1.6 points on its own, because winners feel singled out and work harder all term, whether or not they attend. The reduced form is still 6.4, but only 4.8 of it came through tutoring, so the true effect is 4.8 / 8.0 = 0.60 points per hour. The Wald ratio reports 6.4 / 8.0 = 0.80 and overstates the effect by a third.

Two things follow from that arithmetic. First, the bias is the direct effect divided by the first stage, 1.6 / 8.0 = 0.20 here. So a weak instrument magnifies a violation just as it magnifies noise: the same 1.6-point direct effect with a first stage of 0.8 hours would have added 2.0 to the estimate, not 0.20. Second, the first-stage F does not change at all, because the first stage does not use the outcome. Turn the violation slider up in the widget, and the Wald ratio moves away from the truth while the F stays exactly where it was.

LATE: whose effect is this?

The syllabus names one more idea: LATE, the local average treatment effect. When the effect of X on Y varies from person to person, the ratio does not recover the average effect in the population. It recovers the average among the people whose regressor the instrument actually changed.

Sort the applicants by how they would behave under both states of the lottery. Always-takers get tutoring either way, paying for it elsewhere if the lottery goes against them. Never-takers do not attend even when offered a place. Compliers attend if they win and do not if they lose. Only the compliers change their X when Z changes, so only they contribute to the numerator and the denominator, and the estimate is their average effect. The word "local" in the name means local to that group.

It is easiest to see when the regressor is itself yes-or-no. If 62% of winners attended and 18% of losers did, the first stage is 0.44, and that 0.44 is the complier share. A reduced-form gap of 2.2 points then gives 2.2 / 0.44 = 5.0 points as the effect of attending, for compliers.

Reading 0.44 as the complier share needs one more assumption. The sort above leaves out a fourth type, defiers, who would attend if they lost and stay away if they won. The ratio is the compliers' effect only when there are no defiers. That assumption is called monotonicity, and like exclusion it cannot be tested. Suppose 10% of applicants were defiers. With the same 62% and 18%, compliers would be 54% of applicants and the first stage would still be 0.44. If attending gave compliers 5 points and defiers 10, the reduced form would be 0.54 × 5 − 0.10 × 10 = 1.7. The ratio would then be 1.7 / 0.44 = 3.9 points, below the effect for either group. In a lottery for free places, defiers are hard to imagine. With an instrument that only nudges people, such as the distance to a site, ask whether the nudge could push some of them the other way.

Whether that is the number you wanted depends on the question. A city deciding whether to fund more lottery places is asking precisely about the people extra places would draw in, so the local estimate is the policy-relevant one. A researcher asking what tutoring does for the average applicant is asking something else. A different instrument, changing the behavior of a different group of people, could return a different number, and both numbers would be correct for their own compliers. Report which group your instrument moved, in the same sentence as the estimate.

What no output will tell you

Software prints a first-stage F, so relevance gets checked. Nothing in any package tests independence or exclusion, and no residual plot, diagnostic or goodness-of-fit measure contains information about them. Section 2.7 made the same point about exogeneity, which no residual plot can show. In both cases the data cannot settle the question, so the design has to.

Most two-stage commands can also report a Wu-Hausman test, and R's ivreg prints it beside the first-stage F. It asks whether the two-stage estimate and the ordinary one differ by more than sampling error would explain. One way to compute it is to add the first stage's residuals to the ordinary regression as an extra predictor and test their coefficient. A small p says the two estimates disagree, so if the instrument is valid, the ordinary regression was biased. A large p does not license the ordinary estimate. The test assumes the instrument is valid, which is the open question, and a weak instrument leaves it little power. Set the strength slider in the widget to 0.8, where the first-stage F is typically around 12, and the ordinary regression comes out near 1.04 against a true 0.60. In 10,000 simulated samples at that setting, the test detected the endogeneity at the .05 level only 45% of the time.

What you can do is make the argument in your write-up, where readers can check it. State the instrument and where its variation came from. Report the first stage as a number. Name the direct paths a skeptic would propose, and say what you know that rules each one out. Where the instrument is a lottery or a rule, show that the groups it created look alike on everything you did measure. That does not prove independence, but a clear imbalance would have been strong evidence against it. Then say whose effect the estimate is. A causal claim from observational data rests on its assumptions, so the assumptions belong in the write-up. Reading a Causal-Inference Paper turns the same list into questions to ask of someone else's study, for example in a journal club.