Non-Linear Relationships & Transformations

Fitting a straight line assumes that the relationship is straight. When the relationship bends, the fitted line is wrong in a systematic way, and you can see the pattern in a residual plot. Often a single transformation fixes it, without a bigger model.

The residual plot is where curvature shows up

Fit a line to curved data and it will run above the points at the ends and below them in the middle, or the reverse. On the scatterplot that can be hard to see, because a line through a cloud of points tends to look right. On a plot of residuals against fitted values it is obvious: you get an arch or a valley, not a formless band around zero.

That plot is the first thing Section 2.8 teaches you to draw, and curvature is the violation it catches most reliably. Set the widget's model to the plain line and its truth to any of the curved shapes, then look at the lower panel rather than the upper one.

Two transformations do most of the work

Taking the log of the predictor fits diminishing returns: y climbs quickly at first and then flattens, so equal multiples of x go with equal increases in y. Income against life satisfaction behaves this way, and so does practice against performance.

Taking the log of the outcome fits constant proportional growth: each extra unit of x multiplies y by a fixed factor rather than adding a fixed amount. Populations, compound interest and untreated epidemics do this. Logging the outcome also shortens a long right tail and often makes the variance more even, so it can fix two assumption problems at once.

Adding x², a polynomial term, does not transform either variable. It adds an extra predictor, and it fits a relationship that turns around: rising then falling, or falling then rising. Anxiety against performance is the standard example, and so is any dose that helps up to a point.

The widget's R² readout is always computed on the original scale of y. For the log-outcome model, the predictions are turned back into raw units first. This matters because an R² from a model of log y is not comparable to an R² from a model of y. The two explain variation in different quantities, and software will print both without warning you. One caveat on the back-transform: exponentiating a fitted value estimates the median of y, not its mean, so it runs slightly low. This gap is called retransformation bias, and you should mention it in a write-up whenever the predictions themselves are the point.

Reading a coefficient after a transformation

Write-ups often get this wrong, because the slope no longer means "the change in y per unit of x".

With log x, the slope is the change in y per unit of log x, a unit that is hard to picture. Convert it: doubling x changes y by b₁ × ln 2 ≈ 0.693b₁. If b₁ = 8, every doubling of x adds about 5.5 points to y. A 10% rise in x adds b₁ × ln(1.10) = 0.76. That is close to b₁/10, so the shortcut is safe for small percentages.

With log y, each one-unit rise in x multiplies y by e^b₁. For b₁ = 0.08 that factor is 1.083, an increase of 8.3%. The familiar shortcut, "read 100b₁ as a percentage", gives 8% here and is off by a third of a point. It works well while b₁ is under about 0.1 and gets worse quickly after that: at b₁ = 0.5 the shortcut says 50% and the true figure is 64.9%. So exponentiate, and report the factor.

With x² in the model, neither coefficient means anything on its own, because the effect of x now depends on where you are. Report the turning point, at x = −b₁/(2b₂), and the direction on each side of it. For b₁ = −6 and b₂ = 0.35 the curve reaches its minimum at x = 8.57 and rises after that. Section 2.11 has the same issue: there, too, a coefficient describes the effect at one value of another variable. Mean-centering x before squaring it helps here for the same reason it helps there.

When to transform, and when not to

Transform when the curvature has a shape you can name and a mechanism behind it. Diminishing returns and proportional growth are real mechanisms, and a model that encodes one of them will predict sensibly outside the data you happened to collect. A transformation also keeps the model at one predictor, which keeps the interpretation and the degrees of freedom simple.

Use a larger model when the shape has several bends, when the outcome is a count or a yes/no rather than a quantity, or when the relationship differs by group. A yes/no outcome needs logistic regression, not a log. Counts need a generalized linear model, and a curve with several bends needs a flexible fit, not a different power.

Logs are undefined at zero and below, so a variable containing zeros needs a deliberate decision, not an automatic log(x + 1). Also, choosing a transformation by trying every option and keeping the best R² is a forking path: the winner is partly fitted to noise. Decide from the residual plot and the mechanism.

Going deeper

This lesson covers transformations as part of fitting a regression model. Transformations & Recoding in the Research Toolkit covers the same operations as data preparation: which transformation to apply to a skewed variable, how to recode and bin sensibly, and how to document what you did so the analysis stays reproducible.