Reproducible Workflows

A result you can't reproduce is a result you can't defend. Six months on, a reviewer asks how you got that mean, and "I clicked some menus in SPSS" is not an answer. The fix is a mindset shift: the analysis is the script — a plain-text recipe that turns your raw data into every number in your report, the same way, every time, on anyone's computer.

The analysis is the script

Point-and-click is fine for looking at data; it is a disaster for doing the analysis, because it leaves no record. A click is invisible the moment you make it. A line of code is a permanent instruction you can read, re-run, share, and correct. When your whole pipeline lives in a script:

• it's reproducible: run it again, get the identical result;
• it's correctable: find a bad value, fix the raw file or the code, and re-run to flow the fix through everything downstream in seconds;
• it's auditable: the script is itself the complete log of every decision you made.

None of that survives a session of clicking. You don't need to become a programmer. You need an analysis you can rerun, not one you have to remember.

Order is part of the recipe

A script exposes a subtle danger that clicking hides: the same steps in a different order can give a different answer. Filtering commutes: remove impossible rows then keep group A, or keep group A then remove impossible rows, either way you get the same rows. But filling in missing values does not commute with filtering, because what you fill with depends on which rows are present when you fill. Try to reproduce the target below by fixing the pipeline:

Seeds, structure, and versions

Three cheap habits do most of the work:

• Seed anything random. Bootstrapping, simulation, random train/test splits, and shuffling all use a random-number generator. Set its seed (set.seed(1), np.random.default_rng(1)) at the top of the script and the "random" results become identical on every run. Unseeded randomness is the most common reason a result can't be reproduced, and nothing warns you about it.
• Give the project a skeleton. Lay it out predictably: raw/ for read-only originals, clean/ for script output, scripts/ for the code, output/ for figures and tables. Anyone can then find the pieces and see which direction the data flows. Nothing in output/ is ever hand-made; it's all regenerated by a script.
• Version lightly. You don't need to be a Git wizard, but you do need to stop naming files final_v2_REALfinal.csv. At minimum, keep dated snapshots and never overwrite raw data; better, put the scripts/ folder under version control so every change has a timestamp and a message.

A seed is not enough on its own

Seeding nearly solves the reproducibility problem, but the seed only fixes the sequence given the generator that reads it. Change the generator and the same seed produces different data. This happens in practice: R altered how sample() draws from a seeded stream in version 3.6.0, so set.seed(42); sample(1:10) returns

R 3.6.0 and later :   1  5 10  8  2  4  6  9  7  3
before R 3.6.0    :  10  9  3  6  4  8  5  1  2  7

The seed and the code are the same, but the answer is different. (Draws from rnorm() were not affected by that change, so the problem stays hidden until a random split or a shuffle stops matching.) A supervisor running your script on an older machine gets numbers that don't reproduce yours, and nothing in the script tells either of you why.

So record the environment alongside the seed. It costs one line: sessionInfo() in R prints the R version, the operating system, and the version of every package you loaded, and pip freeze does the same job in Python. Paste that output at the bottom of your script or save it beside the results, and the next person knows what they are trying to match. When a project needs to be reconstructed years later rather than merely described, renv (R) or a requirements.txt (Python) goes further and records the exact package versions so they can be reinstalled.

The last piece is a README in the project root: a short plain-text file saying what the project is, which script to run first, and what each folder holds. It is the one file that is allowed to be written by hand, and it tells a stranger where to start.

The stranger test

One question sums this up: could a stranger, or you two years from now, take your files and reproduce every number, with no email to you and nothing in your head? If the answer is "only if I explain a few things first," those few things aren't documented yet. Everything a result depends on (the raw data, the code, the seed, the package versions, the order) has to live in the project, not in your memory.

Problem 98 of the practice problems shows why a written cleaning log is useful: an assistant recorded every change they made, so you can audit each decision without reverse-engineering a finished file.