Observational Designs
Some of the most important questions in health and social science are about exposures nobody is willing to hand out: smoking, poverty, an infection. What's left is watching instead of intervening, and that is an observational study. You give up the clean causal claim of an experiment, but you gain the ability to study questions that matter most. The craft is knowing which slice of reality each design lets you see, and exactly what it lets you say.
Three ways to watch
All three follow an exposure (a risk factor, a treatment, a habit) and an outcome (a disease, a pass/fail, an event). They differ in when you look and who you start from.
- Cross-sectional: one snapshot in time. You measure exposure and outcome together in a sample, right now. Fast and cheap, great for estimating prevalence, but because both are measured at once, you usually can't tell which came first.
- Cohort (prospective): start with exposed and unexposed people who don't yet have the outcome, then follow them forward and count who develops it. This is the strongest observational design: it measures incidence (new cases over time), so it hands you a risk ratio directly and pins down the time order. The cost: slow, expensive, and hopeless for rare outcomes (you'd follow thousands to see a handful of cases). This is the design behind survival analysis.
- Case-control: flip it around. Start from the outcome: round up people who already have the disease (cases) plus a comparison group who don't (controls), then look backward at who was exposed. Brilliant for rare outcomes and quick to run, but because you chose how many cases and controls to enroll, you can only speak in odds ratios, never risk.
See which slice each design gets
Below, a population of 240 people is followed for 20 years; some develop the outcome, and exposed people are at higher risk. Pick a design and watch which people you actually get to observe, and the estimate each design produces versus the truth.
🔭 Which Slice Does Your Design See?
One fixed population develops an outcome over 20 years. The cohort follows everyone forward; the case-control starts from the cases and samples controls; the cross-sectional sees only a single snapshot. Watch the odds ratio drift away from the risk ratio as the outcome gets more common.
The truth (from the whole population)
What each design estimates (selected design highlighted)
Why case-control speaks in odds ratios
In a cohort you know the denominators (how many exposed and unexposed people you started with), so you can compute a risk (cases ÷ people at risk) and divide them into a risk ratio. In a case-control study you chose the number of cases and controls (often 1:1 or 1:4), so "cases ÷ everyone" is a number you invented, not a real risk. What survives that arbitrary choice is the odds ratio. Happily, the odds ratio from a case-control study estimates the same odds ratio you'd get from the whole population. It's the natural currency of case-control work and of logistic regression.
The odds ratio only approximates the risk ratio when the outcome is rare (the rare-disease assumption). Push the baseline-risk slider up and watch the odds ratio pull away from the risk ratio: when a fifth or a third of people have the outcome, an odds ratio of 2 can mean a risk ratio well under 2. Reporting "twice the odds" as "twice the risk" is one of the most common ways headlines mislead. If you need to put an odds ratio alongside effects reported on other scales, the effect-size converter does the translation.
A fourth way to watch, where the unit is a group
The three designs above all start from a person. A fourth starts from a place. In an ecological study the row in your file is a country, a county, a school or a year, and both the exposure and the outcome arrive as group averages: sugar sold per head against diabetes rates across 40 countries, teacher pay against exam results across school districts. These studies are cheap, they run on data somebody else already collected, and for questions where the exposure genuinely is a group property, such as a law that applies to everyone in a state, the group is the honest unit.
The trap has a name. Concluding something about individuals from group-level associations is the ecological fallacy. The group answer is not merely a noisier version of the individual one; it can point the other way. Take 40 regions of 250 people each, where within any one region screen time and wellbeing are related at r = −.20, while the regions with more screen time on average happen to be the wealthier ones and score higher on average. Aggregate to 40 region means and the correlation is +.77. Test the 10,000 individuals and it is −.10. Same people, same numbers, opposite headline, and the closed-form value and a 400-replication simulation agree to two decimals.
Averaging is what does it. A region mean built from 250 people has almost none of the individual noise left in it, so the between-region pattern is all that survives, magnified. Whatever relationship holds inside regions is averaged away along with the noise. That is why an ecological correlation is usually larger than the individual one and why its sign is not safe to borrow, and it is the same split a multilevel model estimates deliberately, as a within-group slope and a between-group slope that are allowed to disagree.
None of this makes ecological studies useless. They generate hypotheses cheaply and cover populations no cohort could afford, and where the exposure genuinely is a group property they answer the question asked. Report them as what they are: evidence about regions, described in the language of regions. The mirror problem, sampling people in clusters and then wanting to talk about the population, is what sampling methods prices.
Survivorship bias: observational data only contains whoever survived long enough to be in it. Study successful companies and you'll "discover" the traits of the ones that didn't fold; measure patients still in the clinic and you miss the ones who already died or recovered and left. The dangerous part is that the people who dropped out are invisible — the sample looks complete. Always ask: who is missing from this data, and were they missing for a reason tied to the outcome?
Why it matters: observational designs carry most of what we know about diet, disease, education, and policy, because randomizing those exposures is impossible or unethical. But every one estimates a specific quantity (prevalence, incidence, or odds) under specific assumptions, and confounding always lurks (see causal DAGs & confounding). Name your design, name what it can estimate, and name who might be missing from it.
Stats 1 covers the observational-versus-experimental split at course level, alongside the sampling designs that feed it, in Producing Data & Sampling Design.
Common questions
What is the difference between a cohort study and a case-control study?
They run in opposite directions. A cohort study starts from the exposure — it enrolls exposed and unexposed people who don't yet have the outcome and follows them forward to see who develops it, which measures incidence and yields a risk ratio directly. A case-control study starts from the outcome. It rounds up people who already have the disease (cases) plus a comparison group (controls) and looks backward at who was exposed. Cohorts are stronger for establishing time-order but slow and poor for rare outcomes; case-control studies are fast and efficient for rare outcomes but can only estimate an odds ratio.
Isn't a huge observational study better than a small randomized trial?
For precision, yes; for causation, no, and the two are easy to confuse. More data shrinks the confidence interval around whatever you are estimating, but it does nothing to the bias, so a large observational study can deliver a very precise answer to a question you did not mean to ask. Confounding does not average out with sample size the way noise does, which is exactly what the randomization playground shows: the imbalance bars under self-selection stay put as n grows while the randomized ones drift to zero. The honest reading is that the two designs answer different questions. Big observational studies are how we learn what happens in the world as it actually is, including in people a trial would have excluded; trials are how we learn what a change would do. Where they disagree, the trial usually wins on the causal claim and the observational study usually wins on who it applies to.
How do I pick the controls for a case-control study?
Controls have to come from the same population that produced the cases: the people who would have turned up in your case group had they developed the outcome. That single rule disqualifies most convenient choices, because patients recruited on another hospital ward are in hospital for reasons that often travel with the exposure you are studying, which builds the association you are trying to measure. Two practical consequences. Matching controls to cases on a variable buys comparability and spends the variable, since a factor you matched on can no longer be estimated, and matched data then needs a matched analysis (McNemar's test for one binary exposure, conditional logistic regression in general). And enrolling several controls per case pays off quickly then flattens: with 100 cases, a true odds ratio of 2 and 30% exposure among controls, power runs 66% at 1:1, 79% at 2:1, 86% at 4:1 and 88% at 6:1, so the fourth control is usually the last one worth chasing.