Simple Linear Regression
Correlation tells you two variables move together. Regression goes further: it draws the single best straight line through the cloud, so you can actually predict one variable from the other. The whole game is choosing that line, and there's a precise, beautiful rule for what "best" means.
The line and its errors
A regression line is just ŷ = b₀ + b₁x — an intercept and a slope. For any line you propose, each data point sits some vertical distance above or below it. That gap is the residual: the prediction error for that point.
A good line makes those residuals small overall. But small how? Add them up and positives cancel negatives. The fix: square each residual (killing the sign and punishing big misses extra hard), then add them up. That total is the sum of squared residuals (SSR), and the least-squares line is the one, out of all possible lines, that makes it as small as possible.
🎮 Least-Squares Playground
Move the line with the sliders and try to make the error as small as you can. Then hit "best fit" — you won't beat it.
What the slope and intercept mean
- Slope b₁: how much
ŷchanges for each one-unit increase inx. A slope of 0.6 means "every extra point of x buys 0.6 more y, on average." - Intercept b₀: the predicted
ywhenx = 0. Sometimes meaningful, sometimes just where the line crosses the axis.
R²: how much the line explains
Once you've got the best line, R² (which is just the correlation r squared) says what fraction of the variation in y the line accounts for. R² = 0.7 means the line explains 70% of the ups and downs in y; the rest is scatter the line can't capture. Generate new data a few times and watch SSR and R² move together — tighter clouds give smaller SSR and bigger R².
The key idea: "best fit" has an exact meaning: the unique line that minimizes the total squared error. The sliders let you hunt for it; the math finds it instantly. They land in the same place.
Where the name comes from
Francis Galton measured the heights of parents and their grown children in 1886 and found something he called regression toward mediocrity: unusually tall parents had tall children, but on average less tall than themselves. The word stuck to the method, and the effect it names is still one of the easiest ways to fool yourself.
It falls straight out of the slope. Put both variables in standard-deviation units and the least-squares slope becomes exactly r, so a predicted score is always closer to its mean, in SD units, than the predictor was. Take a test whose retest correlation is .70. Of the people who scored 2 SD above the mean on Monday, the group averages 1.4 SD on Friday, and 80% of them score lower the second time. No fatigue, no complacency, no lost form. The same arithmetic runs in reverse for the people who bombed on Monday: they will look improved on Friday, on average, having done nothing.
This is why "we gave extra tutoring to the lowest-scoring students and their scores went up" proves so little on its own. Selecting a group because it sits at an extreme guarantees movement toward the mean at the next measurement, treatment or no treatment. The repair is a design question rather than a statistical one: a control group selected the same way, which quasi-experimental designs exist to arrange. The same shrinkage shows up deliberately in multilevel models, where pulling small groups toward the overall mean is a feature rather than a trap.
Why it matters: simple regression is the foundation of nearly everything in Stats 3 — add more predictors and you've got multiple regression, the workhorse of modern data analysis.
Fit a line to your own data on the Correlation & Regression Calculator — scatterplot, least-squares equation, R², a residual plot, and a copy-ready APA sentence for the slope.
Problem 21 in the practice problems fits a line to that same six-person dataset, predicts a value, and checks the residual against it.
Common questions
How do I know whether my slope is statistically significant?
Divide the slope by its own standard error: t = b / SE, tested against df = n − 2. A slope of b = 2.31 with SE = 0.57 in a sample of 50 gives t(48) = 4.05, p < .001, so the software prints it as significant. Two facts make this easy to sanity-check. Squaring that t gives the model's F (4.05² = 16.4), and testing the slope is the same test as asking whether r differs from 0, so both p-values match exactly. Report the CI alongside it, here [1.16, 3.46], because "significant" only says the interval misses zero, not that the slope is large enough to care about.
What is the difference between correlation and regression?
Correlation gives one symmetric number describing how tightly two variables co-move — no direction, no units. Regression fits an equation (ŷ = b₀ + b₁x) for predicting y from x, with a slope in real units ("each extra study hour buys 2.3 exam points"). Correlation describes; regression predicts.
Can I use my regression line to predict beyond my data's range?
That's extrapolation, and it's where regression goes to die. The line is only supported within the x-values you observed — beyond them, the relationship may bend, flatten, or reverse, and the model gives no warning (a child's growth line predicts 3-meter adults). Predict within the observed range; extrapolate only with strong theoretical justification and loud caveats.