Bias, Blinding & Demand Characteristics
A well-designed study can still give a wrong answer, because bias enters through the people involved. It can come from who ends up in the sample, who drops out, what participants guess you want, or what the researcher hopes to see. Bias is a systematic error. Unlike random noise, it does not average away in a bigger sample, and a thousand extra participants only make it more precise. This lesson covers the common biases, and blinding, the strongest defense against one family of them.
The bias catalog
Six biases cause most of the trouble. Once you can name them, you can design against them:
- Selection bias. The people who end up in your study differ systematically from the population you meant to study. Recruiting gym-goers to study exercise gives you an already-fit sample.
- Attrition bias. People leave the study non-randomly, so the survivors are a skewed remnant. If discouraged participants quit one arm, the two groups are no longer comparable. Non-random dropout is the design-stage face of missing data.
- Response bias. Participants answer in a distorted way, most famously social desirability (answering to look good) on sensitive topics.
- Experimenter-expectancy (observer) bias. The researcher's hopes affect measurement and behavior. The horse Clever Hans "did arithmetic" only because his questioner unconsciously tensed as the correct tap approached.
- Demand characteristics. Participants guess the hypothesis and (helpfully or defiantly) act on their guess rather than naturally.
- Placebo & nocebo effects. Merely believing one is treated changes the outcome, for better (placebo) or worse (nocebo), independent of any active ingredient.
Blinding: keeping people in the dark, on purpose
Blinding (also called masking) hides who is in which condition:
- Single-blind: the participants don't know their condition, which guards against demand characteristics and the placebo effect.
- Double-blind: neither participants nor the researchers who interact with and assess them know the assignment, which also guards against experimenter-expectancy. That is why the "double-blind randomized controlled trial" has such a strong reputation: randomization balances confounds, and double-blinding stops belief and expectation from affecting the results.
But blinding does nothing about who got into the sample or who dropped out. Selection and attrition bias come from the design, not from what anyone knows, and they need better sampling, retention, and intention-to-treat analysis instead. The exercise below sorts the biases into those blinding prevents and those it can't.
Triage the study
Six short study descriptions, each with one main bias. Pick the bias for each one to see the fix. The scoreboard then sorts them into the ones blinding stops and the ones that need a different safeguard.
🩺 Bias Triage & Blinding Scoreboard
For each study, click the dominant bias. Get it right to reveal the fix, and to log whether blinding could have stopped it.
Cases solved: 0 / 6
🕶️ Which biases does blinding stop?
| Bias | Blinding stops it? | The fix |
|---|
The gold standard. A double-blind, placebo-controlled, randomized trial combines three defenses, each against a different bias. Randomization balances confounds, the placebo control isolates the belief effect, and double-blinding stops participants' and researchers' expectations from affecting the result.
Why it matters: statistics cannot correct for bias. A confidence interval quantifies random error, but it is computed as if your measurements were unbiased, so a biased design produces a precise interval around the wrong number. The defenses here are cheap, but they must be built into the design before data collection.
Common questions
How do I blind a study when I can't hide the treatment from participants?
Plenty of interventions can't be masked (surgery, psychotherapy, an exercise program, a classroom method), so a participant always knows which arm they are in. Blinding is not all-or-nothing, though, and the piece that matters most is usually still available: blind the outcome assessor. Whoever scores the symptom rating, codes the recordings, or reads the scans should not know the assignment, which removes experimenter-expectancy even when the participant cannot be fooled. Two more steps help. Use the most objective endpoint you can, a lab value or a task score rather than a clinician's overall impression. And keep whoever analyzes the data blind to the group labels until the analysis is locked. This is the idea behind PROBE designs (Prospective Randomized Open, Blinded Endpoint): open treatment, blinded measurement.
What are demand characteristics?
Demand characteristics are cues in a study that let participants guess its purpose. They then respond to their guess about what's wanted rather than behaving naturally, by trying extra hard in the condition they think should win, or by deliberately doing the opposite. They're a threat to validity because they can create (or erase) an effect that has nothing to do with your manipulation. The defenses are keeping participants blind to the condition and the true aim, using a plausible cover story, and unobtrusive or objective measures.
Does blinding fix every kind of bias?
No. Blinding is specifically a defense against the biases driven by knowing the assignment — the placebo effect, demand characteristics, and experimenter-expectancy. It does nothing about selection bias (who got into the sample), attrition bias (who dropped out non-randomly), or response biases like social desirability. Those are design problems that need better sampling, retention and intention-to-treat analysis, anonymous responding, and so on. Match each bias to its own safeguard rather than hoping blinding covers them all.