Preregistration & Open Science
The replication crisis showed what was going wrong, and open-science practices are the response. The main step is simple: write down exactly what you will do (your hypothesis, your sample size, your analysis, your decision rule) and timestamp it before you see the data. The study then tests a prediction you made in advance, because you can no longer, even unconsciously, choose the analysis after seeing which result it gives.
Confirmatory vs exploratory: both are legitimate
Exploring data is legitimate. The crisis came from disguising exploration as confirmation, one of the questionable research practices. Both modes are valuable, as long as you label each one correctly:
- Confirmatory analysis tests a prediction fixed in advance. It's the only mode in which a p-value means what the textbook says, because the analysis wasn't chosen after seeing the result.
- Exploratory analysis searches the data for patterns to test in a future study. Discoveries often begin this way, but its p-values are descriptive, not decisive, and every finding is a hypothesis awaiting a confirmatory test.
Preregistration doesn't ban exploration. It just draws a bright line between "here's what I predicted and tested" and "here's an intriguing pattern I stumbled on," so readers can weight each appropriately.
Preregistration vs registered reports
Two related tools, differing in when the commitment is reviewed:
- Preregistration: you deposit a timestamped analysis plan (e.g. on the OSF or AsPredicted) before collecting or looking at data. You still submit the finished paper for review afterward as usual.
- Registered report: a journal peer-reviews your plan before the data exist and grants in-principle acceptance: if you carry out the approved design competently, it's published regardless of how the results turn out. This directly neutralizes publication bias, because acceptance no longer depends on getting p < .05.
The rest of the open-science toolkit includes open data and open materials, so others can rerun and reuse your work. It also includes open-access preprints and the "badges" some journals award to signal each practice.
Build a preregistration
Below is a sleep-and-memory study. For each of the eight decisions a good preregistration must pin down, pick the wording, from vague to partial to airtight. Watch the specificity meter. A preregistration works only when a stranger, or a suspicious future you, could not reinterpret it after seeing the data.
📝 Build-a-Preregistration
Eight decisions, three levels of precision each. Aim for a plan with no room left to improvise after the data arrive. (Nothing leaves your browser.)
You can change a preregistered plan. Things go wrong: a lab closes, a measure misbehaves, an assumption fails. You're allowed to deviate, but you have to say so, explain what changed and why, and flag the affected analysis as no longer strictly confirmatory. The plan exists to record which results were predicted. It does not require you to follow it once it no longer fits the situation.
Why it matters: preregistration costs very little and adds a lot of credibility. It is a document you would write anyway, timestamped a few weeks earlier. It protects your readers, your field, and mostly you. It turns "I'm sure I would've predicted that" into a checkable record, and it lets a null result count as a contribution rather than a failure to hide. Design the study, state the hypothesis, lock the plan, and then collect the data.
Problem 97 of the practice problems gives you a manuscript excerpt in which every number is real. You name the four questionable practices in it, including optional stopping and post-hoc exclusions, that a preregistration would have ruled out in advance.
Common questions
Can I preregister a study that analyzes data I already have?
Yes, and it is increasingly expected for secondary-data work, though the timestamp means something different here. With fresh data collection the plan is credible because the numbers do not exist yet. With existing data you have to establish that you had not already seen the part of it that bears on your hypothesis. Good practice is to state exactly what you have looked at, register the analysis before touching the relevant variables, and where possible have someone hold the outcome data until the plan is locked. Some registries (the OSF has a template built for this) ask you to attest to your level of prior access. A preregistration of data you have already explored is not worthless, but it is weaker, and it should be labeled as closer to exploratory than confirmatory.
Are exploratory analyses bad science?
Not at all. Most discoveries begin with exploration. What's harmful is disguising exploration as confirmation: presenting a pattern you found by searching through the data as though you'd predicted it (HARKing). The solution is to label each analysis for what it is. Confirmatory analyses test predictions fixed in advance and have valid p-values. Exploratory analyses generate hypotheses, should be reported as such, and need confirming in a future study. Preregistration draws that line for you.
Does preregistration mean I can never change my analysis?
You can change it, as long as you say so. When something goes wrong (a measure fails, an assumption is violated, a lab closes) you're allowed to deviate. You just have to disclose the change, explain why, and flag the affected analysis as no longer strictly confirmatory. The preregistration gives readers a checkable record of what you predicted and what you decided after seeing the data, so they can weight each appropriately.