Preregistration & Open Science
The replication crisis handed researchers a diagnosis; open-science practices are the treatment. The core move is disarmingly simple: write down exactly what you will do — your hypothesis, your sample size, your analysis, your decision rule — and timestamp it before you see the data. That single act converts a study from a flexible hunt into a genuine test, because you can no longer (even unconsciously) let the data pick the analysis that flatters it.
Confirmatory vs exploratory: both are legitimate
The crisis wasn't caused by exploring data; it was caused by disguising exploration as confirmation, the most common of the questionable research practices. The two modes are both valuable, and the whole game is labeling them honestly:
- Confirmatory analysis tests a prediction fixed in advance. It's the only mode in which a p-value means what the textbook says, because the analysis wasn't chosen after seeing the result.
- Exploratory analysis roams the data looking for patterns worth a future study. It's how discoveries begin, but its p-values are descriptive, not decisive, and every finding is a hypothesis awaiting a confirmatory test.
Preregistration doesn't ban exploration. It just draws a bright line between "here's what I predicted and tested" and "here's an intriguing pattern I stumbled on," so readers can weight each appropriately.
Preregistration vs registered reports
Two related tools, differing in when the commitment is reviewed:
- Preregistration: you deposit a timestamped analysis plan (e.g. on the OSF or AsPredicted) before collecting or looking at data. You still submit the finished paper for review afterward as usual.
- Registered report: a journal peer-reviews your plan before the data exist and grants in-principle acceptance: if you carry out the approved design competently, it's published regardless of how the results turn out. This directly neutralizes publication bias, because acceptance no longer depends on getting p < .05.
Around these sit the rest of the open-science toolkit: open data and open materials so others can rerun and reuse your work, open-access sharing of preprints, and the "badges" some journals award to signal each practice.
Build a preregistration
Below is a sleep-and-memory study. For each of the eight decisions a good preregistration must pin down, pick the wording: from vague, to partial, to airtight. Watch the specificity meter: a preregistration only does its job when a stranger (or a suspicious future you) couldn't wriggle out of it.
📝 Build-a-Preregistration
Eight decisions, three levels of precision each. Aim for a plan with no room left to improvise after the data arrive. (Nothing leaves your browser.)
Preregistration is a plan, not a prison. Reality intervenes — a lab closes, a measure misbehaves, an assumption fails. You're allowed to deviate; you just have to say so, explaining what changed and why, and flag the affected analysis as no longer strictly confirmatory. The commitment buys honesty about which results were predicted, not blind obedience to a plan that no longer fits the world.
Why it matters: preregistration is the cheapest credibility upgrade in all of research: a document you write anyway, timestamped a few weeks earlier. It protects your reader, your field, and mostly you: it turns "I'm sure I would've predicted that" into a checkable record, and lets an honest null result stand as a real contribution instead of a failure to be buried. Design the study, state the hypothesis, lock the plan — then go collect.
Problem 40 of the practice problems hands you a manuscript excerpt in which every number is real, then asks you to name the four questionable practices, optional stopping and post-hoc exclusions among them, that a preregistration would have ruled out in advance.
Common questions
Can I preregister a study that analyzes data I already have?
Yes, and it is increasingly expected for secondary-data work, though the timestamp means something different here. With fresh data collection the plan is credible because the numbers do not exist yet; with existing data you have to establish that you had not already seen the part of it that bears on your hypothesis. Good practice is to state exactly what you have looked at, register the analysis before touching the relevant variables, and where possible have someone hold the outcome data until the plan is locked. Some registries (the OSF has a template built for this) ask you to attest to your level of prior access. A preregistration of data you have already explored is not worthless, but it is weaker, and the honest label for it sits closer to exploratory than confirmatory.
Are exploratory analyses bad science?
Not at all — exploration is how most discoveries begin. What's harmful is disguising exploration as confirmation: presenting a pattern you found by rummaging through the data as though you'd predicted it (HARKing). The honest solution is simply to label each analysis for what it is. Confirmatory analyses test predictions fixed in advance and carry valid p-values; exploratory analyses generate hypotheses and should be reported as such, to be confirmed in a future study. Preregistration draws that line for you.
Does preregistration mean I can never change my analysis?
A preregistration is a plan, not a prison. When reality intervenes (a measure fails, an assumption is violated, a lab closes) you're allowed to deviate; you just have to disclose the change, explain why, and flag the affected analysis as no longer strictly confirmatory. What the commitment buys is an honest, checkable record of what you predicted versus what you decided after seeing the data, not blind obedience to a document, so readers can weight each appropriately.