One Study, Start to Finish: A Complete Worked Project

Courses teach research one stage at a time: design here, cleaning there, ANOVA somewhere else, APA style at the end. Nobody hands you a project and walks it through all of them in order. This guide does that. One question, one design, one file, followed from the sentence you write in your proposal to the paragraph you hand in, with every number computed from memory-2x2.csv in our practice datasets so you can reproduce all of it. It includes the parts a tidy write-up usually hides: a study that turned out smaller than its own power analysis asked for, and four data points a rule flagged as outliers that should not be deleted.

Step 1 · A question you can actually test

Start with the version of the question you'd say out loud: does it matter where you revise? That is a fine motivation and a useless hypothesis, because nothing in it tells you what to measure or what would count as an answer. From Question to Hypothesis is the lesson on closing that gap; the short form is that a testable question names the variables, the population, and the comparison.

Two established findings meet here. Material processed for meaning is remembered better than material processed for surface features, and memory tends to be better when the test happens in the same physical setting as learning. Our study asks whether the second effect depends on the first. That gives a directional prediction worth writing down in advance: reinstating the room helps, but mainly for people who encoded deeply. Predicting an interaction, rather than two separate main effects, changes the design and the sample size, which is why this decision belongs at step one rather than at analysis time.

Then operationalize. Encoding depth becomes a task instruction, shallow being a judgment about a word's appearance and deep being a judgment about its meaning. Context becomes the room used at test, same or different. Memory becomes the number of words recalled from a 30-item list in five minutes. Each of those choices trades some of the concept away for something countable, and Variables & Operationalization works through what gets lost in that trade. Write your choices down before you collect anything, because a definition invented after seeing the data is not a definition, it's a result.

Step 2 · The design, and the number of people it needs

Both factors are manipulated between participants, so every person gives one score in one of four cells: shallow/same, shallow/different, deep/same, deep/different. That is a 2 × 2 between-subjects factorial, and Experiments & Random Assignment covers the random allocation that makes the four groups comparable. A within-subjects version would be more sensitive, but nobody can learn the same word list twice, so Between vs Within Designs settles it for us here.

Now the sample size, and this is where most 2 × 2 proposals go wrong. The intuitive move is to open a power calculator, choose ANOVA, enter four groups and a medium effect of f = .25, and copy the answer. Our calculator gives N = 179 for that, and the number is correct for the question it answers: the omnibus test of whether the four cell means differ at all. Your hypothesis is not that. It is a single-degree-of-freedom test of whether one difference is bigger than another difference, and it needs its own calculation.

So the plan called for 32 people per cell. The file you can download has 25, for the reason most studies fall short: recruitment ended when the term did. At N = 100 the design has 70% power for a medium interaction, and the smallest interaction it could detect 80% of the time is f = .28. That shortfall is not a disaster and it is not a secret. It is a fact about the study that belongs in the limitations section, decided now rather than discovered later. Effect Size & Power covers the logic, and Power Analysis for Complex Designs handles designs where no closed form exists. Lock the plan by preregistering it: hypothesis, design, exclusion rules, and the analysis you intend to run, all timestamped before the first participant.

Step 3 · Meet the data before you analyze it

Download memory-2x2.csv and open it. Four columns, 100 rows: id, encoding, context, recall. Before any test, run the boring checks from The Cleaning Workflow, because each one takes seconds and each one has ruined somebody's thesis:

• Row count and cell count. 100 rows, 25 in each of the four cells. A lopsided design is not fatal, but you want to know about it before the ANOVA, not from it.
• Missing values. None here. Real files are rarely this kind, which is what wellbeing-survey.csv and the cleaning guide are for.
• Duplicate IDs. All 100 are unique.
• Category spelling. encoding takes exactly two values and context exactly two. A stray Deep next to deep would silently become a fifth group.
• Legal range. The list had 30 words, so recall must be a whole number from 0 to 30. Observed range: 2 to 26. Nothing impossible.

Then look at the outcome as a whole. Paste the recall column into the descriptives calculator and you get M = 12.67, SD = 5.41, median 12, quartiles 8 and 16.25, skew 0.28, and no values flagged by the 1.5 × IQR rule. The histogram looks wide and slightly lumpy, which is exactly right for a variable measured across four groups that differ. Judging normality on the pooled outcome is a common mistake; the assumption lives at the level of the cells, and we get to it next.

Step 4 · Assumptions, and four points that are not outliers

A two-way ANOVA asks for independent observations, roughly normal residuals within each cell, and roughly equal variances across cells. The assumptions cheat sheet lists what to check and what to do when a check fails; Assumptions & When They Break is the long version. Independence comes from the design, since each person appears once and was randomly assigned.

Equal variances: the four cell variances are 10.98, 9.58, 10.71 and 12.36, a largest-to-smallest ratio of 1.29. Levene's test on medians gives F(3, 96) = 0.22, p = .886. Nothing to see, which is what a passing check looks like and why you should report it in one clause rather than a paragraph.

Normality: residual skew is 0.12 and excess kurtosis −0.15, both trivially small. One cell, shallow/same, is mildly right-skewed at 0.70. With 25 per cell and equal group sizes, an F test shrugs that off.

Which brings us to the awkward bit. Apply the standard 1.5 × IQR boxplot rule cell by cell, and three cells flag nothing while shallow/same flags four points: recall scores of 3, 15, 16 and 17. A student following a checklist deletes them. Look at why they were flagged first. That cell's quartiles are 8 and 10, so its IQR is 2.0, and the rule's fences sit just 3 points either side of the box. The flag is a statement about an unusually tight middle half, not about extreme scores. Standardized within the cell, those four sit at z = −1.91, 1.71, 2.02 and 2.32. None of them would raise an eyebrow anywhere else in the dataset, and the cell's SD of 3.31 matches its three neighbors almost exactly.

Now the part that decides it. Drop those four rows and rerun the ANOVA: the interaction goes from F(1, 96) = 8.27, p = .005 to F(1, 92) = 12.62, p < .001, and partial η² climbs from .08 to .12. Deleting them makes your hypothesis look better. That is precisely the situation in which a deletion rule must not be invented, and Questionable Research Practices explains how honest people end up there one defensible-feeling step at a time. Outliers: Detect, Investigate, Decide gives the triage that keeps you out of it: a flagged point is removed when it is an error or when it clearly comes from a different population, never because a rule flagged it and never because the analysis improves. These four are plausible recall scores from participants who did the task. They stay. Report the sensitivity check in an appendix, note that the conclusion holds either way, and say which analysis was the preregistered one.

Step 5 · The analysis

Two categorical predictors, one continuous outcome, every participant measured once. That is a two-way between-subjects ANOVA, and if you want to see the reasoning laid out as a decision tree, Plan My Analysis and Which Test? both arrive here. The interactive walkthrough of what the test is doing lives in Factorial ANOVA (Two-Way).

Read that table from the bottom row up. The interaction is significant, so it governs the interpretation, and the two main effects above it become summaries of an effect that is not the same everywhere. Encoding depth has an enormous effect and context has a real one, but saying "reinstating the room helped" as a flat statement would be wrong for half the participants. Whenever an interaction survives, the main effects stop being the headline.

Step 6 · Effect sizes and intervals, which is where the meaning is

A p-value of .005 says the interaction is unlikely under the null and says nothing about its size. Partial η² = .08 does: the interaction accounts for about 8% of the variance left over once encoding and context are accounted for. In Cohen's terms that is f = √(η²_p / (1 − η²_p)) = .29, a shade above the conventional medium. The effect-size converter moves between d, r, η² and f, with one caution worth carrying: the η² it expects is a total η², the share of all the variance, which for this interaction is .03 rather than .08.

The interaction says the effect of context differs by encoding depth. To say how, break it into simple effects, using the pooled error term from the full model so all 96 error df stay available (the same logic as Post-Hoc Tests):

• After shallow encoding, reinstating the room bought 1.12 words, t(96) = 1.20, p = .233, 95% CI [−0.73, 2.97], d = 0.34. The interval includes zero and stretches to nearly three words, so this is a study that cannot tell you whether the shallow effect is null or moderate. Calling it "no effect" would overstate what you have, a distinction Writing About Non-Significant Results takes seriously.
• After deep encoding, it bought 4.92 words, t(96) = 5.27, p < .001, 95% CI [3.07, 6.77], d = 1.49. Large by any convention, and the interval excludes zero comfortably.

Report the intervals, not just the tests. A confidence interval carries the estimate, the uncertainty and the test result in one object, and it is the only one of the three that tells a reader what your study could not rule out. Confidence Intervals is the lesson if that claim feels strong.

Step 7 · The paragraph you hand in

Everything above compresses into a few sentences. From Output to Results Section gives the ordering rule that most write-ups violate: descriptives first, then the omnibus test, then the decomposition, then the effect sizes, and no interpretation of what any of it means for theory. Save that for the discussion.

The typography carries meaning and reviewers notice it. Latin letters for statistics are italic (F, p, t, M, d), Greek letters are not (η²), p and d take no leading zero because they cannot exceed one in the relevant sense, and p = .000 is never a real value: below .001 you write p < .001. Build any of these sentences in the APA Results Formatter, which applies the rules for you and recomputes p from your statistic and df to check the numbers agree. The APA cheat sheet is the printable version, and Reporting Statistics in APA Style the lesson.

One figure earns its place here: the four cell means with error bars, encoding on the x-axis and one line per context. Non-parallel lines are the interaction, and readers grasp it faster from the picture than from η². Tables & Figures covers the formatting.

Step 8 · Limitations, and what a reviewer will ask

A limitations section written honestly is a strength, and Discussion & Limitations makes the case at length. Four belong in this study, and three of them were decided long before the analysis:

• The sample was smaller than planned. 25 per cell against a target of 32, giving 70% power for a medium interaction. State the achieved sample, the target, and the assumed effect size. Do not compute power from the effect you observed: post-hoc power is a rearrangement of your own p-value and adds nothing, which is why the figure appears in this guide's source comment and not in its results.
• The non-significant shallow effect is not evidence of absence. The interval [−0.73, 2.97] is consistent with no effect and with a moderate one. Say that, rather than "context did not matter for shallow encoders".
• Between-subjects designs pay for their cleanliness. Individual differences in memory ability land in the error term, which is part of why the interaction needed so many people.
• Generalization. Word lists in two rooms, with whoever a university term supplies. Sampling Methods is the honest treatment of who your results are actually about.

Then the ethics review, which is not a form to be survived. Informed Consent & Ethics Committees sets out the paperwork; the substance is what a committee would ask about this specific study. Did participants agree to a memory test knowing recall would be scored, and were they told they could stop? Were they told which room condition they were in, and if the cover story omitted it, does that count as deception requiring a debrief? Are the recall scores stored with the ID that links them to a name, and if so, where does that key live and when is it destroyed? Privacy & Confidentiality is the lesson, and the answer for a file like ours is that id should be a code with the linking key held separately and deleted on schedule.

The last question is the one that makes all the earlier steps worth taking. Could a stranger holding your raw file and your written procedure arrive at the same numbers you published? For this project the answer is yes, because you can check it right now on the same CSV we used. That is the standard everything in this guide was aiming at.

Keep going

• Practice Datasets: memory-2x2.csv to redo this project yourself, and seven other files with their own worked solutions.
• Analyze Your Thesis Data in JASP: the same journey with the software clicks shown.
• Choose the Right Statistics: for when your design is not a 2 × 2.
• From SPSS Output to APA Results: which number on the screen goes in which slot.
• Clean Your Survey Data: the step this project got to skip.
• Practice Problems: worked problems that make you produce the numbers by hand.