Data Entry & Validation
Every dataset starts with somebody typing numbers into cells, and humans typing numbers make mistakes: an extra digit, a wrong unit, a value pasted one row too low. A single impossible value can flip a mean or sink a whole analysis. Validation is the habit of catching those errors the moment they enter, with rules that never get tired or distracted.
Where errors come from
- Typos. A slipped finger turns an age of 41 into 411, or a decimal wanders (74.2 → 742).
- Unit mix-ups. Height entered in inches into a centimeters column; weight in pounds among kilograms. The number looks fine; it's just measuring the wrong thing.
- Copy-paste drift. A column pasted one cell high or low, so every value after a point belongs to the wrong person.
- Inconsistent categories.
F,Female,Femal,f(with a trailing space) all meant the same thing, and now count as four groups.
Validation rules: three kinds
You don't hunt for errors by re-reading every cell. You write down what a legal value looks like and let the rule find the illegal ones:
- Range checks: a numeric field must fall inside plausible bounds, such as
0 ≤ age ≤ 120or150 ≤ reaction_ms ≤ 3000. - Allowed-value sets: a categorical field must be one of a fixed list, e.g.
sex ∈ {M, F, Other}. Anything else is a typo. - Cross-field logic: two columns must agree, e.g.
birth_year = current_year − age; a "pregnant" flag can't beYesfor a male record.
Two more habits catch what rules miss: double entry (type the data twice, from two people or two passes, then compare, and mismatches are errors) and the plausibility mindset behind checks like GRIM: always asking "could this number even happen?" A mean of 3.14 from 7 integer ratings can't; an adult who weighs 7 kg can't either.
Time to hunt. This 40-row dataset has 8 planted impossible values. Click every cell you think is wrong, against the clock, then reveal the validation rules that would have caught each one automatically.
🔍 Spot-the-Error Speed Round
Eight cells hold values that can't be real. Click each one you find — a green tick confirms a real error, a red flash means that value is fine. The clock starts on your first click.
The lesson of the reveal: every error you found (and every one you missed) maps to a rule you could have written before collecting a single row. That's the shift: from proofreading data by eye, which never scales past a few dozen rows, to declaring the rules once and letting them police thousands. Validation flags the impossible; a value that is unusual but genuinely possible is a separate decision, taken up in outliers in practice.
Validate at the door, not at the deadline. An impossible value is cheapest to fix the instant it's typed, when you can still check the source. Found six months later in the middle of analysis, the same value is a mystery — was it 41 or 14? — that often can't be recovered at all.
Why it matters: validation protects everything downstream: one 511 hiding in an age column drags the mean, inflates the SD, and can masquerade as an influential outlier in a model. Point-of-entry tools help (a spreadsheet's Data → Data Validation, or SPSS: Data → Validate Data), but the durable version is a short script of range, set, and cross-field checks you rerun every time the data changes, like the R/Python one below. Pair it with a clear codebook, which is where the legal values are defined in the first place.
Common questions
My data came from an online form, not hand entry. Do I still need validation?
Yes; the errors just change shape. Dropdowns and required fields stop many typos, but they do nothing about a respondent who straightlines every scale, enters a height in the wrong unit, or submits the form twice, and automated exports add their own failure modes: a shifted header row, a silent encoding change, an ID that arrives as text on one download and a number on the next. The same three checks still apply, all written from your codebook (range, allowed-value set, cross-field logic), plus a few aimed at the source: look for duplicate submissions, impossibly fast completion times, and columns whose type changed between one export and the next. Whatever produced the file, you validate what actually arrived, not what you assume the tool guaranteed.
Should I just delete impossible values I find?
A validation flag is the start of an investigation, not a license to delete. First find out what happened: an impossible value is usually a fixable data-entry error, so check the source and correct it (an age of 511 was almost certainly 51). If it truly can't be recovered, mark it as missing and say so — don't silently drop the whole row. Deleting values hides problems and can bias your results; every change should be documented and reversible.
What is double data entry?
It's entering the same data twice (by two people, or by one person on two separate passes) and then comparing the two versions cell by cell. Wherever they disagree, at least one entry is wrong, so the mismatches point you straight to the typos to check against the source. It sharply cuts the undetected-error rate and is standard in clinical trials and other high-stakes data collection. For smaller projects, validation rules plus a careful proofreading pass are a lighter-weight substitute.