Observational Designs
Some of the most important questions in health and social science are about exposures no researcher can ethically assign: smoking, poverty, an infection. Then you can only watch, and a study that watches without intervening is an observational study. You give up the clean causal claim of an experiment, but you can study the questions that matter most. The skill is knowing which people each design lets you observe, and what it lets you conclude.
Three ways to watch
All three follow an exposure (a risk factor, a treatment, a habit) and an outcome (a disease, a pass/fail, an event). They differ in when you look and who you start from.
- Cross-sectional: one snapshot in time. You measure exposure and outcome together in a sample, right now. Fast and cheap, great for estimating prevalence, but because both are measured at once, you usually can't tell which came first.
- Cohort (prospective): start with exposed and unexposed people who don't yet have the outcome, then follow them forward and count who develops it. This is the strongest observational design: it measures incidence (new cases over time), so it gives a risk ratio directly and establishes the time order. The cost: it is slow, expensive, and impractical for rare outcomes (you'd follow thousands to see a handful of cases). This is the design behind survival analysis.
- Case-control: flip it around. Start from the outcome: round up people who already have the disease (cases) plus a comparison group who don't (controls), then look backward at who was exposed. Efficient for rare outcomes and quick to run, but because you chose how many cases and controls to enroll, you can only estimate odds ratios, never risk.
See which slice each design gets
Below, a population of 240 people is followed for 20 years; some develop the outcome, and exposed people are at higher risk. Pick a design and watch which people you actually get to observe, and the estimate each design produces versus the truth.
🔭 Which Slice Does Your Design See?
One fixed population develops an outcome over 20 years. The cohort follows everyone forward; the case-control starts from the cases and samples controls; the cross-sectional sees only a single snapshot. Watch the odds ratio drift away from the risk ratio as the outcome gets more common.
The truth (from the whole population)
What each design estimates (selected design highlighted)
Why case-control speaks in odds ratios
In a cohort you know the denominators (how many exposed and unexposed people you started with), so you can compute a risk (cases ÷ people at risk) and divide them into a risk ratio. In a case-control study you chose the number of cases and controls (often 1:1 or 1:4), so "cases ÷ everyone" is a number you set yourself, not a real risk. The odds ratio does not depend on that choice, and the odds ratio from a case-control study estimates the same odds ratio you'd get from the whole population. It is the standard measure in case-control work and in logistic regression.
The odds ratio only approximates the risk ratio when the outcome is rare (the rare-disease assumption). Push the baseline-risk slider up and watch the odds ratio pull away from the risk ratio. When a fifth or a third of people have the outcome, an odds ratio of 2 can mean a risk ratio well under 2. Reporting "twice the odds" as "twice the risk" is one of the most common ways headlines mislead. If you need to put an odds ratio alongside effects reported on other scales, the effect-size converter does the translation.
A fourth way to watch, where the unit is a group
The three designs above all start from a person. A fourth starts from a place. In an ecological study each row in your file is a country, a county, a school or a year, and both the exposure and the outcome are group averages. Examples are sugar sold per head against diabetes rates across 40 countries, or teacher pay against exam results across school districts. These studies are cheap and use data somebody else already collected. When the exposure is a group property, such as a law that applies to everyone in a state, the group is the right unit of analysis.
Concluding something about individuals from group-level associations is the ecological fallacy. The group answer is not merely a noisier version of the individual one. It can point the other way. Take 40 regions of 250 people each. Within any one region, screen time and wellbeing are related at r = −.20. But the regions with more screen time on average happen to be the wealthier ones, and they score higher on wellbeing on average. Aggregate to 40 region means and the correlation is +.77. Correlate the 10,000 individuals and it is −.10.
The averaging causes this. A region mean built from 250 people has almost none of the individual noise left in it, so only the between-region pattern remains, and it looks stronger. Whatever relationship holds inside regions is averaged away along with the noise. So an ecological correlation is usually larger than the individual one, and you cannot assume the individual correlation has the same sign. A multilevel model estimates the two parts separately, as a within-group slope and a between-group slope that are allowed to disagree.
None of this makes ecological studies useless. They generate hypotheses cheaply and cover populations no cohort could afford, and where the exposure is a group property they answer the question asked. Report them as evidence about regions, described in the language of regions. The reverse problem, sampling people in clusters and then drawing conclusions about the population, is covered in sampling methods.
Survivorship bias: observational data only contains whoever survived long enough to be in it. Study successful companies and you'll "discover" the traits of the ones that didn't fold. Measure patients still in the clinic and you miss the ones who already died or recovered and left. The people who dropped out are invisible, so the sample looks complete. Always ask: who is missing from this data, and were they missing for a reason tied to the outcome?
Why it matters: most of what we know about diet, disease, education, and policy comes from observational designs, because randomizing those exposures is impossible or unethical. But each one estimates a specific quantity (prevalence, incidence, or odds) under specific assumptions, and confounding is always possible (see causal DAGs & confounding). Name your design, name what it can estimate, and name who might be missing from it.
Stats 1 covers the observational-versus-experimental split at course level, alongside the sampling designs that feed it, in Producing Data & Sampling Design.
Common questions
What is the difference between a cohort study and a case-control study?
They run in opposite directions. A cohort study starts from the exposure. It enrolls exposed and unexposed people who don't yet have the outcome and follows them forward to see who develops it, which measures incidence and gives a risk ratio directly. A case-control study starts from the outcome. It rounds up people who already have the disease (cases) plus a comparison group (controls) and looks backward at who was exposed. Cohorts are stronger for establishing time-order but slow and poor for rare outcomes. Case-control studies are fast and efficient for rare outcomes but can only estimate an odds ratio.
Isn't a huge observational study better than a small randomized trial?
For precision, yes; for causation, no, and the two are easy to confuse. More data shrinks the confidence interval around whatever you are estimating, but it does nothing to the bias. A large observational study can therefore give a very precise answer to a question you did not mean to ask. Confounding does not average out with sample size the way noise does. The randomization interactive shows this: the imbalance bars under self-selection stay put as n grows, while the randomized ones drift to zero. The two designs answer different questions. Big observational studies show what happens in the world as it is, including in people a trial would have excluded. Trials show what a change would do. Where they disagree, the trial is usually more trustworthy on the causal claim, and the observational study on who the result applies to.
How do I pick the controls for a case-control study?
Controls have to come from the same population that produced the cases: the people who would have turned up in your case group had they developed the outcome. That single rule disqualifies most convenient choices. Patients recruited on another hospital ward, for example, are in hospital for reasons that are often related to the exposure you are studying, which distorts the association you are trying to measure. Two practical points follow. Matching controls to cases on a variable makes them comparable on it, but you can no longer estimate that variable's effect. Matched data then needs a matched analysis (McNemar's test for one binary exposure, conditional logistic regression in general). And extra controls per case raise power quickly at first, then level off. With 100 cases, a true odds ratio of 2 and 30% exposure among controls, power is 66% at 1:1, 79% at 2:1, 86% at 4:1 and 88% at 6:1, so beyond four controls per case the gain is small.