One Question, an ANOVA and a Regression

A great many intermediate courses end the same way. Four of you collect your own data on one question, analyze it as a one-way ANOVA across at least three conditions, analyze the same relation again as a multiple regression with control variables, and hand in a paper. Every stage of that has a lesson on this site and none of them is the route. This guide is the route, walked once on commute-mood.csv from our practice datasets, so every number below can be reproduced from the file you download. The interesting part turns up at step 7, where the two analyses have to be reconciled and one of the group differences loses nearly half its size.

Step 1 · A question your design can actually answer

A project question has to survive two tests before anyone collects anything. It needs a predictor with three or more levels, because a two-level predictor gives you a t-test and the assignment asks for an ANOVA. And it needs an outcome measured on a continuous scale, because that is what both analyses expect. From Question to Hypothesis is the lesson on turning a vague interest into something testable; Variables & Operationalization is the one on deciding what you will actually measure.

The question carried through this guide: does how students travel to campus relate to how they feel when they arrive? The predictor is commute mode with three levels (walking or cycling, public transport, car). The outcome is a mood rating taken on arrival, on a 0 to 100 slider.

Now the harder half. Nobody was randomly assigned to a commute mode, so any difference you find has an obvious rival explanation, and a reviewer will name it before you finish your sentence. People who live near campus walk. People who live far away do not. If distance also affects mood, then a comparison of the three modes is partly a comparison of how far people live from campus, and you cannot tell the two apart by staring at the group means. That rival variable is a confounder, and it is the reason the assignment asks for a regression as well as an ANOVA: the regression is where you hold it constant.

Two controls were chosen before any data existed, for two different reasons:

• Distance from campus in kilometers, because it plausibly causes both the predictor and the outcome. Where you live is settled long before you pick a commute mode, so distance is upstream of mode rather than downstream of it. Controlling for something that sits after your predictor in the causal chain answers a different question, which Multiple Regression takes some care over.
• Hours slept the night before, because it plausibly affects mood without having much to do with commute mode. A control like this buys precision rather than correcting bias, and as you will see below it does exactly that.

Write both of these into the proposal with the reason attached. A control variable defended after the analysis reads as a fishing trip, and Questionable Research Practices explains why that reputation is deserved. Preregistration & Open Science is the tidy way to make the commitment public.

Step 2 · Collecting it, and the sample size that decides everything

This is a short campus survey: intercept students on arrival, ask for the three variables and the mood rating, four fieldworkers over a fortnight. Designing Surveys & Questionnaires covers the wording; Sampling Methods covers the part that limits your conclusions, since whoever walks past your table at 9 a.m. is not a random sample of anything.

One design decision settles the analysis and cannot be revisited later. Each person contributes one row, and nobody appears in two conditions, so this is a between-subjects design and the ANOVA is the between-subjects kind. Had you followed the same people across modes, you would need Repeated Measures ANOVA instead, with an entirely different error term. Between vs Within Designs is worth ten minutes before you print anything.

Then the sample size, from the power calculator. For a one-way ANOVA with k = 3 groups, a medium effect (f = .25), α = .05 and 80% power, you need N = 158, about 53 per group. The study in this file collected 150, which is 50 per group and slightly short: at that size the design has 78% power for a medium effect, and the smallest effect it detects at 80% power is f = .26. That is worth one honest sentence in the method section and one in the limitations. It is not worth computing power from the effect you ended up observing, which is a rearrangement of your own p-value and tells a reader nothing they did not already have.

Step 3 · The codebook, then the cleaning

Write the codebook before the first row arrives. It is a page that says what every column is called, what type it is, what values are legal and what a blank cell means, and it is what lets a stranger open your file next year and understand it. Codebooks & Documentation makes the case at length. Here is the whole of ours:

Cleaning then has something to check against, which is the entire point of writing it first. Run the four passes The Cleaning Workflow sets out: structure (150 rows, five columns, one row per person), completeness (no blank cells), validity (every mode is one of the three legal strings, every number inside its declared range, participant_id unique), and consistency. Our file passes all four, which is a property of a file built for teaching rather than of fieldwork; Data Entry & Validation shows what a real intake sheet looks like, and the messy-clinic.csv in the same dataset library is the version with the dirt left in.

Never edit the raw file. Save the original untouched and do every correction in a script or a syntax file, so the path from what you collected to what you analyzed is visible to anyone who asks. Reproducible Workflows is the lesson, and it is also the answer to the question a supervisor asks when two group members report different means.

Step 4 · Look at the data before you test anything

Descriptives first, by condition, because an ANOVA table is unreadable without them. Paste a column into the descriptives calculator and it returns the mean, the sample SD, the quartiles, a t-based interval and the outlier flags:

Read the covariate columns as carefully as the outcome column, because they are the argument for the whole second analysis. Walkers live 2.09 km from campus and drivers live 9.80 km away, a gap of nearly eight kilometers, so the three groups are not comparable on distance and never were. Sleep, by contrast, is flat across the three (6.88, 6.89, 7.03). Predict from that pair of facts alone what each control will do to the group differences, then check yourself at step 7.

The whole-sample outcome is roughly symmetric: M = 67.32, SD = 8.68, Mdn = 68, and an adjusted Fisher–Pearson skewness of 0.01. One value trips the 1.5 × IQR rule, participant 81's mood of 44, which sits just under the lower fence of 45.5. Outliers: Detect, Investigate, Decide is emphatic about what happens next, and it is not deletion. Check the row against the codebook: 44 is a legal mood value, the person's distance and sleep are both ordinary, nothing suggests a typing error. Keep it, and check later whether it moved anything. It does not; at step 6 the model puts its residual at 2.5 standard errors below zero, which in 150 cases is unremarkable, and its Cook's distance is 0.03 against a threshold of 1.

Step 5 · The one-way ANOVA

Two assumptions decide whether the ordinary F is the right test, and ANOVA & t-Test Assumptions is the lesson on both. Independence comes from the design: one row per person, no clustering, nobody surveyed twice. Equal variances gets tested, and Levene's test on the three mood distributions gives F(2, 147) = 0.51, p = .604. The group variances are 61.5, 63.6 and 49.9, a largest-to-smallest ratio of 1.28, so the pooled error term is honest and ordinary F is fine. Had that test come back significant you would switch to Welch's ANOVA, which the same lesson covers.

Then the table. SPSS prints it under Analyze → Compare Means → One-Way ANOVA; the arithmetic behind every cell is in One-Way ANOVA, whose Build the ANOVA Table widget will walk it through cell by cell on your own numbers.

Read it left to right and every number has a job. The two sums of squares add to the total exactly, which is the partition the whole method rests on. The mean squares are sums of squares divided by their own degrees of freedom, and the within-groups one, 58.367, is the pooled variance; its square root, 7.64, is the pooled standard deviation s_p that every follow-up test below divides by. F is the ratio of the two mean squares, 1325.340 / 58.367 = 22.71, against a critical F*(2, 147) of 3.06 from the printed tables. And η² = .236 says commute mode accounts for about 24% of the variance in arrival mood, with a 90% confidence interval of [.14, .32].

The omnibus test says the three means are not all equal and stops there. Since the question named a specific comparison in advance, the follow-up should be a planned contrast rather than a fishing expedition through all three pairs. Active travel against the average of the two motorized modes has coefficients [1, −½, −½], and Post-Hoc Tests builds it interactively:

ψ̂ = 8.91, SE = 1.32, t(147) = 6.73, p < .001, 95% CI [6.30, 11.53]. Active commuters arrive about nine mood points above the motorized average, and the interval is nowhere near zero. Written as whole numbers, [2, −1, −1], the estimate doubles to 17.82 and so does its standard error, leaving t untouched, which is the cheapest check that you built the contrast correctly.

The three pairwise comparisons, Bonferroni-corrected against a critical t** of 2.42, tell the same story with more detail: walking beats public transport by 9.12 points and beats car by 8.70, both with adjusted p < .001, while public transport and car differ by 0.42 points, adjusted p = 1.000. Two groups and one group, not three.

Step 6 · The same question, asked as a regression

A regression needs numbers, and mode is a word. Dummy coding is the translation: pick one level as the reference, then make one 0/1 column for each of the others. With public transport as the reference you get walk_cycle (1 if they walked, else 0) and car. Two columns for three groups, never three, because a third would be perfectly predictable from the other two and the model would refuse to fit. Categorical Predictors & Dummy Coding is the lesson, and SPSS will do it for you under Transform → Recode into Different Variables or through the Automatic Linear Modeling dialog.

Fit the dummies on their own first. It is one line in the write-up and it is the anchor for everything that follows.

Every one of those numbers is already on this page. The constant, 64.140, is the mean of the reference group. The walk coefficient, 9.120, is exactly 73.26 − 64.14, and the car coefficient is 64.56 − 64.14. R² equals η² to fifteen decimal places, and the model F is the ANOVA's F. There is no new analysis here, only a different notation for the one you already ran, which Inference for Regression unpacks.

Now add the controls, which is the model the assignment is really asking for.

Say each coefficient out loud with the phrase that makes it true. Holding distance and sleep constant, active commuters arrive 5.09 points above transport users. Holding everything else constant, each additional kilometer from campus costs 0.76 mood points, and each additional hour of sleep is worth 3.43. Drop the phrase and you have said something the model does not support; it is not decoration, it is the meaning of a partial coefficient.

Three diagnostics before you trust any of it, all from Regression Diagnostics and Assumptions of Regression. The residual plot (tick Save → Unstandardized residuals in SPSS, then plot them against the fitted values) should be a shapeless band, and this one is: the correlation between absolute residual and fitted value is −.02, so there is no fanning out. The residuals are near enough normal, with a skewness of −0.15 and an excess kurtosis of −0.28, and no case has leverage over the 3p/n threshold of 0.10 or a Cook's distance anywhere near 1. The largest is 0.05. VIF is the last column of the table above, and Multicollinearity & Variable Selection gives the reading: a VIF of 2.58 on distance means its standard error is √2.58 = 1.6 times what it would be if distance were unrelated to the other predictors. That is the price of controlling for something correlated with your predictor, it is visible and modest, and everything here sits comfortably under the conventional threshold of 5.

Step 7 · Making the two analyses agree

You now have to explain, in the paper, why the same comparison is worth 9.12 points in one table and 5.09 in another. Both numbers are correct and they answer different questions.

The 9.12 is the raw difference between two groups of real students. The 5.09 is what is left of it once you compare people who live equally far from campus and slept equally long. The gap between them is not error. It has a name, omitted-variable bias, and Multiple Regression gives its size exactly, as an identity that holds in your sample rather than merely on average:

b_raw = b_controlled + b_control × δ

δ is how much the control itself differs between the groups, which you get by regressing the control on the same dummies. Walkers live 5.35 km closer to campus than transport users (δ = −5.35) and sleep essentially the same amount (δ = −0.004). Put the numbers in:

5.092 + (−0.756 × −5.348) + (3.432 × −0.004) = 9.120

which is the raw coefficient to the last decimal the table prints, and to fourteen decimals in the file. So the arithmetic answers its own question. About 4.03 of the raw 9.12-point advantage of walking was distance wearing a commute-mode costume, and 0.01 of it was sleep, which is what you should have predicted from step 4 when sleep came out flat across the three groups. A control that is balanced across conditions cannot shift a coefficient; it can only shrink the standard errors, and it did, which is why the model's residual standard error fell from 7.64 to 6.83 and R² rose from .236 to .398.

The car coefficient makes the same point in the opposite direction. Raw, drivers sat 0.42 points above transport users; controlled, they sit 1.70 above. Drivers live furthest from campus, so the raw comparison was holding them back, and neither version is statistically significant in any case. Report both and say which one your conclusion rests on.

One honest limit on all of this. Holding a confounder constant is not the same as randomizing, and no regression can control for a variable you did not measure. The claim you can defend is that the mode difference is not merely a distance difference; the claim you cannot defend is that commuting by bicycle causes better moods. Observational Designs and Quasi-Experiments & Natural Experiments are the lessons on where that line sits.

Step 8 · The paper

Four sections, in the order The IMRaD Structure sets out, and the results section is the only one this guide can write for you. Its ordering rule is the one most drafts get wrong: descriptives, then the omnibus test, then the follow-ups, then the effect sizes, and no discussion of what any of it means. From Output to Results Section is the lesson, and the APA Results Formatter will build any of these sentences with the typography applied and recompute the p from your statistic as a check.

Arrival mood differed across the three commute modes, F(2, 147) = 22.71, p < .001, η² = .24, 90% CI [.14, .32]. A planned contrast comparing active travel with the two motorized modes was significant, ψ̂ = 8.91, SE = 1.32, t(147) = 6.73, p < .001, 95% CI [6.30, 11.53]. Bonferroni-corrected pairwise comparisons showed that walking or cycling exceeded public transport, M_diff = 9.12, t(147) = 5.97, p < .001, and car use, M_diff = 8.70, t(147) = 5.69, p < .001, while public transport and car use did not differ, M_diff = −0.42, t(147) = −0.275, p = .784.

A multiple regression predicting arrival mood from commute mode alone reproduced the ANOVA, R² = .24, F(2, 147) = 22.71, p < .001. Adding distance from campus and hours slept improved the fit, R² = .40, F(4, 145) = 24.00, p < .001, and reduced the advantage of active travel over public transport from 9.12 to 5.09 points, b = 5.09, SE = 1.79, t(145) = 2.84, p = .005, 95% CI [1.55, 8.64]. Both covariates predicted mood: each additional kilometer from campus was associated with a 0.76-point decrease, b = −0.76, SE = 0.22, t(145) = −3.48, p < .001, and each additional hour of sleep with a 3.43-point increase, b = 3.43, SE = 0.64, t(145) = 5.38, p < .001.

The typography is load-bearing and reviewers notice it. Latin statistics are italic (F, t, p, b, M, R²), Greek ones are not (η², ψ̂), and p, η² and R² take no leading zero. Below .001 you write p < .001 and never p = .000. Reporting Statistics in APA Style is the lesson and the APA cheat sheet is the printable version to keep beside you.

One figure earns its place. Plot the three group means with 95% confidence intervals on the outcome scale, since that is the comparison the whole paper turns on, and a reader gets the overlap between transport and car from the picture faster than from the p-value. A second panel showing the controlled estimates beside the raw ones makes step 7 visible in a way three paragraphs cannot. Tables & Figures That Don't Lie covers the formatting rules, and the chart chooser is there if you are unsure of the shape.

Limitations, then. Three of the four were fixed long before the analysis, which is the usual pattern. The sample fell 8 people short of the power target and was collected wherever fieldworkers happened to stand, so it represents the students who walk past a table on a weekday morning. Nobody was randomly assigned to a commute mode, so the causal reading stays shut. And mood was one slider at one moment, which Reliability & Validity would call a thin measure of a broad construct. Write those plainly; Discussion & Limitations makes the case that a limitations section done honestly reads as competence rather than weakness.

The ethics review is not a form to be survived either. Students gave their commute mode, their home distance and their sleep, which together can identify someone on a small campus, so the file that leaves your laptop should carry a code rather than a name and the key should live somewhere else and be destroyed on schedule. Privacy & Confidentiality is the lesson; Informed Consent & Ethics Committees is the paperwork.

Step 9 · What a supervisor will grade

Nine things, in the order they get looked at. Most projects that lose marks lose them in the first three, which were decided before any analysis existed.

Then the test that covers all nine at once. Hand your raw file and your written method to somebody outside the group and ask whether they can arrive at your numbers. For this project the answer is yes, and you can check it yourself right now on the same CSV. The Final Checklist is the version to run the night before submission.

Keep going

• Practice Datasets: commute-mood.csv to redo this project yourself, with nine other files and their worked solutions.
• One Study, Start to Finish: the same journey for a 2 × 2 factorial experiment rather than a survey.
• From SPSS Output to APA Results: which number on the screen belongs in which slot.
• Clean Your Survey Data: the cleaning step worked on a file that needs it.
• Choose the Right Statistics: for when your design is not three conditions and an outcome.
• Practice Problems: ANOVA tables and regression coefficients to work by hand.