Non-Linear Relationships & Transformations

A straight line is an assumption, not a fact about your data. When the relationship bends, the fitted line is wrong in a patterned way, and the pattern is visible in a plot you should already be drawing. Often the repair is one transformation rather than a bigger model.

The residual plot is where curvature shows up

Fit a line to curved data and it will run above the points at the ends and below them in the middle, or the reverse. On the scatterplot that can be hard to see, because the eye is generous about lines. On a plot of residuals against fitted values it is unmistakable: instead of a formless band around zero you get an arch or a valley.

That plot is the first thing Section 2.8 teaches you to draw, and curvature is the violation it catches most reliably. Set the widget's model to the plain line and its truth to any of the curved shapes, then look at the lower panel rather than the upper one.

Two transformations do most of the work

Taking the log of the predictor fits diminishing returns: y climbs quickly at first and then flattens, so equal multiples of x buy equal amounts of y. Income against life satisfaction behaves this way, and so does practice against performance.

Taking the log of the outcome fits constant proportional growth: each extra unit of x multiplies y by a fixed factor rather than adding a fixed amount. Populations, compound interest and untreated epidemics do this. Logging the outcome also shrinks a long right tail and often steadies the variance at the same time, which is why it repairs two assumption problems with one move.

Adding x² is not a transformation of either variable but an extra predictor, and it fits a relationship that turns around: rising then falling, or falling then rising. Anxiety against performance is the standard example, and so is any dose that helps up to a point.

Note what the widget's R² readout does: it is always computed on the original scale of y, with the log-outcome model's predictions turned back into raw units first. That is deliberate. An R² from a model of log y is not comparable to an R² from a model of y, because the two are explaining variation in different quantities, and software will happily print both without warning you.

Reading a coefficient after a transformation

This is the part that goes wrong in write-ups, because the slope no longer means "the change in y per unit of x".

With log x, the slope is the change in y per unit of log x, and nobody thinks in those. Convert it: doubling x changes y by b₁ × ln 2 ≈ 0.693b₁. If b₁ = 8, every doubling of x is worth about 5.5 points of y. A 10% rise in x is worth b₁ × ln(1.10) = 0.76, which is close enough to b₁/10 that the shortcut is safe for small percentages.

With log y, each one-unit rise in x multiplies y by e^b₁. For b₁ = 0.08 that factor is 1.083, an increase of 8.3%. The familiar shortcut, "read 100b₁ as a percentage", gives 8% here and is off by a third of a point. It stays serviceable while b₁ is under about 0.1 and degrades quickly after that: at b₁ = 0.5 the shortcut says 50% and the true figure is 64.9%. Exponentiate rather than approximating, and report the factor.

With x² in the model, neither coefficient means anything on its own, because the effect of x now depends on where you are. What you report is the turning point, at x = −b₁/(2b₂), and the direction on each side of it. For b₁ = −6 and b₂ = 0.35 the curve bottoms out at x = 8.57 and rises after that. The same "a coefficient is conditional now" logic runs through Section 2.11, and mean-centering x before squaring it helps here for the same reason it helps there.

When to transform, and when not to

Transform when the curvature has a shape you can name and a story behind it. Diminishing returns and proportional growth are real mechanisms, and a model that encodes one of them will predict sensibly outside the data you happened to collect. A transformation also keeps the model at one predictor, which keeps the interpretation and the degrees of freedom simple.

Reach for something larger when the shape has several bends, when the outcome is a count or a yes/no rather than a quantity, or when the relationship differs by group. A yes/no outcome belongs in logistic regression rather than under a log, counts belong in a generalized linear model, and a curve that wanders needs a flexible fit rather than a cleverer power.

Two warnings worth carrying. Logs are undefined at zero and below, so a variable containing zeros needs a considered decision rather than a reflexive log(x + 1). And a transformation chosen by trying every option and keeping the best R² is a forking path, not an analysis, so decide from the residual plot and the mechanism instead of from the leaderboard.

Going deeper

This is the course-level treatment, aimed at the regression model in front of you. Transformations & Recoding in the Research Toolkit takes the same operations as a data-preparation job: which transformation to apply to a skewed variable, how to recode and bin sensibly, and how to document what you did so the analysis stays reproducible.