Data Entry & Validation
Every dataset starts with somebody typing numbers into cells, and humans typing numbers make mistakes: an extra digit, a wrong unit, a value pasted one row too low. A single impossible value can flip a mean or sink a whole analysis. Validation is the habit of catching those errors the moment they enter, with rules that never get tired or distracted.
Where errors come from
- Typos. A slipped finger turns an age of 41 into 411, or a decimal wanders (74.2 → 742).
- Unit mix-ups. Height entered in inches into a centimeters column; weight in pounds among kilograms. The number looks fine; it's just measuring the wrong thing.
- Copy-paste drift. A column pasted one cell high or low, so every value after a point belongs to the wrong person.
- Inconsistent categories.
F,Female,Femal,f(with a trailing space) all meant the same thing, and now count as four groups.
Validation rules: three kinds
You don't hunt for errors by re-reading every cell. You write down what a legal value looks like and let the rule find the illegal ones:
- Range checks: a numeric field must fall inside plausible bounds, such as
0 ≤ age ≤ 120or150 ≤ reaction_ms ≤ 3000. - Allowed-value sets: a categorical field must be one of a fixed list, e.g.
sex ∈ {M, F, Other}. Anything else is a typo. - Cross-field logic: two columns must agree, e.g.
birth_year = current_year − age; a "pregnant" flag can't beYesfor a male record.
Two more habits catch what rules miss. Double entry means typing the data twice, by two people or in two passes, and comparing: every mismatch is an error. The plausibility mindset behind checks like GRIM means always asking "could this number even happen?" A mean of 3.15 from 7 integer ratings can't, and neither can an adult who weighs 7 kg.
Time to hunt. This 40-row dataset has 8 planted impossible values. Click every cell you think is wrong, against the clock, then reveal the validation rules that would have caught each one automatically.
🔍 Spot-the-Error Speed Round
Eight cells hold values that can't be real. Click each one you find: a green tick confirms a real error, a red flash means that value is fine. The clock starts on your first click.
Every error you found (and every one you missed) breaks a rule you could have written before collecting a single row. Proofreading by eye doesn't scale past a few dozen rows, but rules written once can check thousands. Validation flags the impossible. A value that is unusual but possible is a separate decision, covered in outliers in practice.
Validate as the data comes in. An impossible value is cheapest to fix the instant it's typed, when you can still check the source. Found six months later in the middle of analysis, the same value is a mystery (was it 41 or 14?) that often can't be recovered at all.
Why it matters: validation protects everything downstream: one 511 hiding in an age column pulls the mean up, inflates the SD, and can look like an influential outlier in a model. Point-of-entry tools help (a spreadsheet's Data → Data Validation, or SPSS: Data → Validate Data). The lasting solution, though, is a short script of range, set, and cross-field checks that you rerun every time the data changes, like the R/Python one below. Pair it with a clear codebook, where the legal values are defined in the first place.
Common questions
My data came from an online form, not hand entry. Do I still need validation?
Yes. The errors are just different. Dropdowns and required fields stop many typos, but they do nothing about a respondent who straightlines every scale, enters a height in the wrong unit, or submits the form twice. Automated exports add their own failure modes: a shifted header row, a silent encoding change, an ID that arrives as text on one download and a number on the next. The same three checks still apply, all written from your codebook (range, allowed-value set, cross-field logic). Add a few aimed at the source: look for duplicate submissions, impossibly fast completion times, and columns whose type changed between one export and the next. Whatever produced the file, validate what actually arrived, not what you assume the tool guaranteed.
Should I just delete impossible values I find?
A validation flag is the start of an investigation, not a license to delete. First find out what happened: an impossible value is usually a fixable data-entry error, so check the source and correct it (an age of 511 was almost certainly 51). If it can't be recovered, mark it as missing and say so, rather than silently dropping the whole row. Deleting values hides problems and can bias your results; every change should be documented and reversible.
What is double data entry?
It's entering the same data twice (by two people, or by one person on two separate passes) and then comparing the two versions cell by cell. Wherever they disagree, at least one entry is wrong, so the mismatches point you straight to the typos to check against the source. It sharply cuts the undetected-error rate and is standard in clinical trials and other high-stakes data collection. For smaller projects, validation rules plus a careful proofreading pass are a lighter-weight substitute.