Reproducible Workflows

A result you can't reproduce is a result you can't defend. Six months on, a reviewer asks how you got that mean, and "I clicked some menus in SPSS" is not an answer. The fix is a mindset shift: the analysis is the script — a plain-text recipe that turns your raw data into every number in your report, the same way, every time, on anyone's computer.

The analysis is the script

Point-and-click is fine for looking at data; it is a disaster for doing the analysis, because it leaves no record. A click is invisible the moment you make it. A line of code is a permanent instruction you can read, re-run, share, and correct. When your whole pipeline lives in a script:

• it's reproducible: run it again, get the identical result;
• it's correctable: find a bad value, fix the raw file or the code, and re-run to flow the fix through everything downstream in seconds;
• it's auditable: the script is itself the complete log of every decision you made.

None of that survives a session of clicking. The goal isn't to be a programmer; it's to make your analysis a thing that can be re-run rather than re-remembered.

Order is part of the recipe

A script exposes a subtle danger that clicking hides: the same steps in a different order can give a different answer. Filtering commutes: remove impossible rows then keep group A, or keep group A then remove impossible rows, either way you land in the same place. But filling in missing values does not commute with filtering, because what you fill with depends on which rows are present when you fill. Try to reproduce the target below by fixing the pipeline:

Seeds, structure, and versions

Three cheap habits carry most of the reproducibility load:

• Seed anything random. Bootstrapping, simulation, random train/test splits, and shuffling all use a random-number generator. Set its seed (set.seed(1), np.random.default_rng(1)) at the top of the script and the "random" results become identical on every run. Unseeded randomness is the classic silent irreproducibility.
• Give the project a skeleton. A predictable folder layout — raw/ (read-only originals), clean/ (script output), scripts/ (the code), output/ (figures, tables) — means anyone can find the pieces and see which direction the data flows. Nothing in output/ is ever hand-made; it's all regenerated by a script.
• Version lightly. You don't need to be a Git wizard, but you do need to stop naming files final_v2_REALfinal.csv. At minimum, keep dated snapshots and never overwrite raw data; better, put the scripts/ folder under version control so every change has a timestamp and a message.

A seed is not enough on its own

Seeding feels like the end of the reproducibility problem, and it very nearly is, but the seed only fixes the sequence given the generator that reads it. Change the generator and the same seed produces different data. This is not hypothetical: R altered how sample() draws from a seeded stream in version 3.6.0, so set.seed(42); sample(1:10) returns

R 3.6.0 and later :   1  5 10  8  2  4  6  9  7  3
before R 3.6.0    :  10  9  3  6  4  8  5  1  2  7

Same seed, same one line of code, different answer. (Draws from rnorm() were untouched by that change, which is why the problem stays hidden until the day a random split or a shuffle stops matching.) A supervisor running your script on an older machine gets numbers that don't reproduce yours, and nothing in the script tells either of you why.

So record the environment alongside the seed. It costs one line: sessionInfo() in R prints the R version, the operating system, and the version of every package you loaded, and pip freeze does the same job in Python. Paste that output at the bottom of your script or save it beside the results, and the next person knows what they are trying to match. When a project needs to be reconstructed years later rather than merely described, renv (R) or a requirements.txt (Python) goes further and records the exact package versions so they can be reinstalled.

The last piece is a README in the project root: a short plain-text file saying what the project is, which script to run first, and what each folder holds. It is the one file that is allowed to be written by hand, and it is what turns a correct folder of files into a folder a stranger can actually enter.

The stranger test

The single question that captures all of this: could a stranger — or you, two years from now — take your files and reproduce every number, with no email to you and nothing in your head? If the answer is "only if I explain a few things first," those few things aren't documented yet. Everything a result depends on (the raw data, the code, the seed, the package versions, the order) has to live in the project, not in your memory.

Problem 41 of the practice problems shows why a written cleaning log earns its keep: an assistant recorded every change they made, so you can audit each decision instead of reverse-engineering a finished file.