One Study, Start to Finish: A Complete Worked Project

Courses teach research one stage at a time: design here, cleaning there, ANOVA somewhere else, APA style at the end. This guide follows one project through all of those stages in order, from the sentence in your proposal to the paragraph you hand in. Every number is computed from memory-2x2.csv in the practice datasets, so you can reproduce all of it. The guide also covers two problems that a tidy write-up usually hides. The study ended up smaller than its power analysis asked for, and a rule flagged four data points as outliers that should not be deleted.

Step 1 · A question you can actually test

Start with the question as you would say it out loud: does it matter where you revise? That is a good motivation but not yet a hypothesis, because it does not say what to measure or what would count as an answer. A testable question names the variables, the population, and the comparison. From Question to Hypothesis shows how to get there.

Two established findings are relevant. Material processed for meaning is remembered better than material processed for surface features. And memory tends to be better when the test happens in the same physical setting as learning. This study asks whether the second effect depends on the first. That gives a directional prediction to write down in advance: reinstating the room helps, but mainly for people who encoded deeply. Predicting an interaction, rather than two separate main effects, changes the design and the sample size, so make this decision at step one and not at analysis time.

Then operationalize. Encoding depth becomes a task instruction: shallow means judging a word's appearance, and deep means judging its meaning. Context becomes the room used at test, same or different. Memory becomes the number of words recalled from a 30-item list in five minutes. Each choice captures only part of the concept, and Variables & Operationalization explains what each kind of choice leaves out. Write your choices down before you collect anything. A definition chosen after seeing the data can be bent to fit the results.

Step 2 · The design, and the number of people it needs

Both factors are manipulated between participants, so every person gives one score in one of four cells: shallow/same, shallow/different, deep/same, deep/different. That is a 2 × 2 between-subjects factorial, and Experiments & Random Assignment covers the random allocation that makes the four groups comparable. A within-subjects version would be more sensitive, but a person cannot learn the same word list twice, so the design has to be between-subjects (Between vs Within Designs explains the trade-off).

Next comes the sample size, where many 2 × 2 proposals go wrong. The obvious approach is to open a power calculator, choose ANOVA, enter four groups and a medium effect of f = .25, and copy the answer. The power calculator gives N = 179 for that. The number is correct, but it is for the omnibus test of whether the four cell means differ at all. Your hypothesis is about a different test: a single-degree-of-freedom test of whether one difference is bigger than another. That test needs its own calculation.

So the plan called for 32 people per cell. The file you can download has 25, for a common reason: recruitment ended when the term did. At N = 100 the design has 70% power for a medium interaction, and the smallest interaction it could detect 80% of the time is f = .28. The shortfall does not invalidate the study, and it should not be hidden: report it in the limitations section. Effect Size & Power covers the logic, and Power Analysis for Complex Designs covers designs with no closed-form power formula. Preregister the plan before the first participant: hypothesis, design, exclusion rules, and the analysis you intend to run, all with a timestamp.

Step 3 · Meet the data before you analyze it

Download memory-2x2.csv and open it. Four columns, 100 rows: id, encoding, context, recall. Before any test, run the basic checks from The Cleaning Workflow. Each takes seconds, and skipping any of them can ruin a thesis:

• Row count and cell count. 100 rows, 25 in each of the four cells. Unequal cells are not fatal, but you should know about them before you run the ANOVA.
• Missing values. There are none here. Real files usually have some, and wellbeing-survey.csv and the cleaning guide show how to handle them.
• Duplicate IDs. All 100 are unique.
• Category spelling. encoding takes exactly two values and context exactly two. A stray Deep next to deep would silently become a fifth group.
• Legal range. The list had 30 words, so recall must be a whole number from 0 to 30. Observed range: 2 to 26. Nothing impossible.

Then look at the outcome as a whole. Paste the recall column into the descriptives calculator and you get M = 12.67, SD = 5.41, median 12, quartiles 8 and 16.25, skew 0.28, and no values flagged by the 1.5 × IQR rule. The histogram is wide and slightly lumpy, as expected for a variable measured across four groups with different means. Do not judge normality from the pooled outcome. The assumption applies within each cell, and Step 4 checks it there.

Step 4 · Assumptions, and four points that are not outliers

A two-way ANOVA assumes independent observations, roughly normal residuals within each cell, and roughly equal variances across cells. The assumptions cheat sheet lists what to check and what to do when a check fails, and ANOVA & t-Test Assumptions covers them in full. Independence comes from the design, since each person appears once and was randomly assigned.

Equal variances: the four cell variances are 10.98, 9.58, 10.71 and 12.36, a largest-to-smallest ratio of 1.29. Levene's test on medians gives F(3, 96) = 0.22, p = .886. The variances are similar, so report this check in one clause, not a paragraph.

Normality: residual skew is 0.12 and excess kurtosis −0.04, both very small. One cell, shallow/same, is mildly right-skewed at 0.70. With 25 per cell and equal group sizes, the F test is robust to skew of that size.

Next, outliers. Apply the standard 1.5 × IQR boxplot rule cell by cell, and three cells flag nothing while shallow/same flags four points: recall scores of 3, 15, 16 and 17. A student following a checklist might delete them. First look at why they were flagged. That cell's quartiles are 8 and 10, so its IQR is 2.0 and the rule's fences are only 3 points beyond each end of the box. The flags reflect an unusually narrow middle half, not extreme scores. Standardized within the cell, the four points are at z = −1.91, 1.71, 2.02 and 2.32. None of these scores would look unusual in any other cell, and the cell's SD of 3.31 is almost the same as the other three cells'.

Then check what deleting them would do. Drop those four rows and rerun the ANOVA: the interaction goes from F(1, 96) = 8.27, p = .005 to F(1, 92) = 12.62, p < .001, and partial η² rises from .08 to .12. Deleting them makes your hypothesis look better. This is the situation in which a deletion rule must not be invented after the fact. Questionable Research Practices explains how researchers with no intent to cheat end up doing so, one reasonable-seeming step at a time. Outliers: Detect, Investigate, Decide gives the rule to follow instead. Remove a flagged point only when it is an error or clearly comes from a different population, never because a rule flagged it and never because the analysis improves. These four are plausible recall scores from participants who did the task, so they stay. Report the sensitivity check in an appendix, note that the conclusion holds either way, and say which analysis was preregistered.

Step 5 · The analysis

Two categorical predictors, one continuous outcome, and every participant measured once: that calls for a two-way between-subjects ANOVA. Plan My Analysis and Which Test? both reach the same answer if you want to see the reasoning as a decision tree. Factorial ANOVA (Two-Way) explains what the test does, with an interactive.

Read the table from the bottom row up. The interaction is significant, so interpret it first. The two main effects above it then average over an effect that differs between conditions. Encoding depth has a very large effect and context has a smaller but clear one, but saying "reinstating the room helped" without qualification would be wrong for half the participants.

Step 6 · Effect sizes and intervals, which is where the meaning is

A p-value of .005 says that an interaction this large would be unlikely if the null were true. It says nothing about how large the interaction is. Partial η² = .08 does: the interaction accounts for about 8% of the variance not explained by the two main effects. In Cohen's terms that is f = √(η²_p / (1 − η²_p)) = .29, slightly above the conventional medium of .25. The effect-size converter converts between d, r, η² and f. Note that it expects a total η², the share of all the variance, and for this interaction that is .03, not .08.

The interaction says the effect of context differs by encoding depth. To say how, break it into simple effects, using the pooled error term from the full model so all 96 error df stay available (the same logic as Post-Hoc Tests):

• After shallow encoding, reinstating the room added 1.12 words, t(96) = 1.20, p = .233, 95% CI [−0.73, 2.97], d = 0.34. The interval includes zero and extends to nearly three words, so the study cannot tell you whether the shallow effect is zero or moderate. Calling it "no effect" would claim more than the data show, and Writing About Non-Significant Results explains how to word it.
• After deep encoding, it added 4.92 words, t(96) = 5.27, p < .001, 95% CI [3.07, 6.77], d = 1.49. That is large by any convention, and the interval is well clear of zero.

Report the intervals, not only the tests. A confidence interval shows the estimate, its uncertainty and the test result at once. Unlike a p-value or a point estimate on its own, it also shows which effect sizes your study could not rule out. Confidence Intervals explains this in more detail.

Step 7 · The paragraph you hand in

All of the above fits in a few sentences. From Output to Results Section gives the order to follow: descriptives first, then the omnibus test, then the simple effects, then the effect sizes. Leave what it means for theory to the discussion.

Reviewers notice typography. Latin letters for statistics are italic (F, p, t, M, d), and Greek letters are not (η²). Statistics that cannot exceed 1, such as p and η², take no leading zero. And p = .000 is never a real value: below .001, write p < .001. Build any of these sentences in the APA Results Formatter, which applies the rules for you and recomputes p from your statistic and df to check that the numbers agree. The APA cheat sheet is the printable version, and Reporting Statistics in APA Style the lesson.

Add one figure: the four cell means with error bars, encoding on the x-axis and one line per context. Non-parallel lines are the interaction, and readers see it faster in the picture than in η². Tables & Figures covers the formatting.

Step 8 · Limitations, and what a reviewer will ask

A specific, accurate limitations section strengthens a thesis, and Discussion & Limitations explains why. This study has four limitations, and three of them were fixed before the analysis began:

• The sample was smaller than planned. 25 per cell against a target of 32, giving 70% power for a medium interaction. State the achieved sample, the target, and the assumed effect size. Do not compute power from the effect you observed. Post-hoc power is a transformation of your own p-value and adds no information.
• The non-significant shallow effect is not evidence of absence. The interval [−0.73, 2.97] is consistent with no effect and with a moderate one. Say that, rather than "context did not matter for shallow encoders".
• Between-subjects designs avoid carryover but lose power. Individual differences in memory ability are part of the error term, and that is one reason the interaction needed so many people.
• Generalization. The study used word lists in two rooms, with whichever students volunteered that term. Sampling Methods explains how to describe the population your results apply to.

Then the ethics review. Informed Consent & Ethics Committees sets out the paperwork. The substance is what a committee would ask about this specific study. Did participants agree to a memory test knowing recall would be scored, and were they told they could stop? Were they told which room condition they were in, and if the cover story omitted it, does that count as deception requiring a debrief? Are the recall scores stored with the ID that links them to a name, and if so, where does that key live and when is it destroyed? For a file like this one, id should be a code, with the key that links codes to names stored separately and deleted on schedule. Privacy & Confidentiality covers this.

The last question is reproducibility. Could someone with your raw file and your written procedure get the same numbers you published? For this project the answer is yes: you can check it now on the same CSV.

Keep going

• Practice Datasets: memory-2x2.csv to redo this project yourself, and nine other files with their own worked solutions.
• One Question, an ANOVA and a Regression: the same steps for a survey with three conditions and control variables, rather than a 2 × 2 experiment.
• Analyze Your Thesis Data in JASP: the same steps, with the software clicks shown.
• Choose the Right Statistics: for when your design is not a 2 × 2.
• From SPSS Output to APA Results: which number on the screen goes in which slot.
• Clean Your Survey Data: the step this project could skip.
• Practice Problems: worked problems that make you produce the numbers by hand.