Confidence Intervals
A single estimate — "the average is 102" — pretends to a precision it doesn't have. A confidence interval is more honest: instead of one number, it reports a range of plausible values, like "somewhere between 99 and 105." But the phrase "95% confident" trips almost everyone up. Let's pin down what it actually means.
From estimate to interval
Recall that a sample mean has a sampling distribution with a known spread (the standard error). A confidence interval just wraps a margin around the estimate:
estimate ± (critical value) × (standard error)
Pick a higher confidence level and the critical value grows, so the interval gets wider — you trade precision for a better chance of being right. The playground makes that trade-off, and the real meaning of "95%," impossible to miss. (To build one from your own numbers rather than a slider, paste a column into the descriptives calculator, which returns the interval along with everything that goes into it; the tables calculator supplies the critical value on its own.)
🎮 Confidence Interval Coverage
The dashed line is the true mean (normally unknown — here we cheat and show it). Each horizontal bar is one study's interval. Green bars caught the truth; red ones missed.
What "95% confident" really means
Keep drawing studies and watch the capture rate settle near your chosen level. That is the meaning of confidence:
"95% confidence" is a statement about the procedure, not about any single interval. If you repeated the study endlessly, about 95% of the intervals you'd build would contain the true value. Any one interval either contains it or doesn't — there's no "95% probability" hiding inside a single result.
This is why the red bars matter. At 95% confidence, roughly 1 interval in 20 misses entirely — and that's not a bug, it's the advertised failure rate. Switch to 99% and misses become rare, but every interval gets wider (less precise). Switch to 90% and intervals tighten, but you miss more often.
What changes the width
- Confidence level ↑ → wider. More certainty demands more room.
- Sample size ↑ → narrower. More data shrinks the standard error, so the margin shrinks. Slide n up and watch the bars tighten around the truth.
Why it matters: confidence intervals are how good science reports uncertainty — not a false-precision point estimate, but a range plus a stated reliability. They're also two sides of the same coin as hypothesis tests, which come next.
Problem 8 of the practice problems builds a 95% interval from 25 students by hand, then shows exactly how much quadrupling the sample narrows it.
Common questions
Does a 95% confidence interval mean there's a 95% chance the true value is inside?
Not quite — the 95% describes the procedure, not any single interval. If you repeated the study endlessly, 95% of the intervals built this way would capture the truth; the one interval in front of you either contains it or doesn't. If you want a statement like "95% probability the parameter is in this range," that's what Bayesian credible intervals provide.
Can I tell whether two groups differ by seeing if their confidence intervals overlap?
Half of that inference is safe and the other half is not. If two 95% intervals do not overlap, the difference is significant at the .05 level, always. The reverse does not follow: overlapping intervals are perfectly compatible with a significant difference, because the interval around a difference is narrower than the two individual intervals side by side. For two equally precise means, the difference reaches p = .05 while the intervals still overlap by about 29% of their length, so a whole band of genuinely significant results looks non-significant to the eyeball test. Test the difference itself, or plot the interval around the difference rather than two intervals around the means. For paired or repeated measures the eyeball test is worse still, since the two intervals ignore the within-person correlation that the test uses.
What does it mean if my confidence interval contains zero?
For a difference or an effect, an interval like [−1.2, +3.5] says "zero effect is among the plausible values" — so the result is not statistically significant at the matching α. That's a statement of uncertainty, not proof of no effect: the interval also contains plenty of non-zero values. Wide intervals containing zero usually mean the study was underpowered.