Regression Diagnostics
You can fit a regression line to any cloud of points and get a slope, an intercept, and an R². None of that tells you whether a straight line was the right model. To find out, you look at what the line leaves unexplained: the residuals.
The residual plot
A residual is the gap between an actual point and the line's prediction. Plot the residuals against the fitted values to check the model. You want to see no pattern at all: a formless, random cloud centered on zero. Any pattern in the residuals means the model missed something.
🎮 Residual Plot Reader
Top: data with the least-squares line. Bottom: residuals vs. fitted values. Switch the scenario and learn each tell-tale shape.
Data & fitted line
Residuals vs. fitted (you want a shapeless cloud around zero)
The residual signatures: a shapeless cloud = healthy. A curve / U-shape = the true relationship is nonlinear (a straight line is wrong, so add a curve term). A fan or funnel = non-constant variance (heteroscedasticity), which makes the standard errors wrong. A lone point far from the rest = an influential outlier that may be tilting the line on its own.
The five things to check
- Linearity. The residual cloud has no curve. If it bends, the relationship isn't a straight line.
- Constant variance. The vertical spread of residuals is the same all the way across. A funnel means it isn't.
- Normal residuals. Best checked with a Q-Q plot of the residuals; matters most for small samples.
- Independent residuals. One row's error should tell you nothing about the next row's. This is the assumption the plot above cannot show you, and the one that does the most damage when it fails.
- Influential points. No single observation should dominate the fit. Leverage and influence are not the same thing, and the regression assumptions lesson separates them. An unusual predictor value gives a point leverage, but only leverage together with a large residual moves the line much. Cook's distance measures that combination.
Independence: the check the plot can't make
A plot of residuals against fitted values can look perfectly healthy on data that breaks independence, because the problem is in the order or the grouping of the rows, not in their spread. Two kinds of dependence are common. Measurements taken in sequence (daily sales, weekly symptom scores) tend to be followed by similar values. This is called autocorrelation, and you see it by plotting the residuals against time or row number. The Durbin-Watson statistic your software prints is the formal test: values near 2 mean no autocorrelation, and values near 0 mean each residual is positively correlated with the one before it.
The second kind is clustering: several rows from the same participant, class, clinic, or family. Ordinary regression treats every row as independent evidence, so ten measurements from one person count as ten people. Standard errors and p-values come out too small. Unlike the other four problems on this list, nothing you do to the residuals afterwards repairs it. You need a model that accounts for the grouping, such as mixed and multilevel models.
What to do about it
A curved pattern? Transform a variable or add a polynomial/nonlinear term. A funnel? Transform the outcome (logs often help) or use robust/weighted standard errors. An influential outlier? Investigate it (is it a data-entry error, or a real-but-rare case?) and report the fit with and without it.
Why it matters: a model with the wrong form can still have a high R², so a high R² does not mean the model is right. A residual plot takes a few seconds to read and can stop you from trusting a model that doesn't fit. The same checks apply in multiple regression, and the assumption-checks sheet lists them all on one printable page.
Problem 43 of the practice problems shows a fan-shaped residual plot, one case with leverage 0.31 and Cook’s D 0.68, and two predictors correlated at .92. It asks what to fix before you read any coefficient.
Common questions
How do I know whether a pattern in my residual plot is real or just noise?
Random residuals look patterned far more often than people expect. With thirty points, clumps and gentle curves show up constantly in data generated from a perfectly linear model, and if you look for structure you will find some. A quick check is a lineup. Simulate several residual plots from a model where the assumptions hold by construction, mix your real plot in among them, and see whether you can pick yours out. If you can't, whatever worried you is within the range of ordinary noise. This is the lineup protocol of Buja and colleagues (2009). Running it informally two or three times trains your eye better than any threshold.
What is heteroscedasticity in simple terms?
It means the spread of the residuals changes across the range of predictions. The usual shape is a funnel, with small errors at small fitted values and large errors at large ones (income data often looks like this). The slope estimate stays unbiased, but its standard errors and p-values become unreliable. To fix it, transform y (a log is the usual choice) or use robust (heteroscedasticity-consistent) standard errors.
Should I delete outliers from my regression?
Not automatically. First investigate. A data-entry error (age = 250) gets fixed or removed, but a real, extreme case is information about the world. For real outliers, check influence (Cook's distance), then report the model with and without the point. If the conclusion depends on one observation, report that as a finding.