Mixed & Multilevel Models (GLMMs)

Data is often grouped: students within schools, patients within hospitals, repeated measurements within people. Those observations aren't independent, and pretending they are breaks your standard errors. Mixed (multilevel) models handle grouping head-on, and they come with a genuinely magical trick: groups borrow strength from each other.

Fixed effects vs. random effects

A mixed model has two kinds of terms. Fixed effects are the usual coefficients: effects assumed constant across the whole population. Random effects let each group have its own intercept (and sometimes its own slope), drawn from a distribution the model estimates. Instead of estimating every group in isolation, the model learns "how much do groups vary?" and uses that to inform each one.

How much of the variation is the grouping?

Two spreads come out of a fitted mixed model: τ (tau), the standard deviation between group means, and σ (sigma), the spread of individuals inside a group. Their ratio has a name you will meet everywhere clustered data is discussed. The intraclass correlation is ICC = τ² / (τ² + σ²), the share of the total variation that lives between groups rather than within them, and equivalently the correlation you expect between two observations drawn from the same group.

In the playground below, σ is fixed at 11 and the τ slider is therefore an ICC dial. At its default τ = 8 the ICC is .35, so a third of the variation is between groups. Slide τ down to 1 and it collapses to .008, which is the regime where groups barely differ and shrinkage pulls everything to the grand mean. Push τ to 30 and it reaches .88, where each group is nearly its own population. The same number governs how badly clustering hurts a survey, which is why sampling methods uses it to compute the price of cluster sampling.

What ignoring the grouping costs

The assumptions poster tells you to rule out "hidden clustering" before you trust an ordinary test, which is easy to nod at and hard to act on. Clustering is hidden whenever the rows of your file are not the units you sampled: pupils inside the classes you recruited, trials inside the people you ran. Analyzing those rows as though each were a fresh participant has a name, pseudoreplication, and it has a price you can compute in advance.

Survey statisticians got there first. The design effect from sampling methods, deff ≈ 1 + (m − 1)ρ, converts your row count into the sample size those rows are really worth: m observations per cluster and an ICC of ρ. Twenty classes of twenty pupils is 400 rows, but at an ICC of .10 the design effect is 2.9, so those rows carry about as much information as 138 independent pupils. Run a test that believes in all 400 and the standard error is too small by a factor of √2.9, which is 1.7.

Simulate that and the damage is plain. With the treatment given to whole classes and no true effect at all, an ordinary two-sample t-test across the 400 pupils calls the difference significant 24.7% of the time. At an ICC of .05 it is 16.0% and at .20 it is 37.4%. Average each class into a single number first, so that the 20 units in the test are the 20 units that were randomized, and the false-positive rate returns to 4.8%. A five-percent test that fires a quarter of the time is most of the reason cluster-randomized trials are analyzed the way they are.

The direction of the damage depends on where the effect lives. Split the treatment inside every class instead, so each class contributes both conditions, and the same neglected clustering makes the ordinary test slightly conservative rather than reckless: 4.1% false positives, and about two points of power given away (70.9% against the mixed model's 73.2% for a difference of a quarter of a standard deviation). Between-cluster comparisons are inflated by ignored clustering. Within-cluster comparisons are merely blunted by it, which is the same arithmetic that makes a paired test beat an unpaired one on the same people counted twice.

How would you know? Fit the model with no predictors at all, read off τ and σ, and look at the ICC. Anything much above zero means the grouping is carrying real variance and the rows are not independent. A design where an ICC of .02 would be alarming is a design with very large clusters, since it is m and ρ together, not ρ alone, that decide the price.

The three ways to handle groups

• Complete pooling: ignore the groups entirely; one estimate for everyone. Simple, but throws away real group differences.
• No pooling: estimate each group on its own. Honest, but tiny or noisy groups get wild, unreliable estimates.
• Partial pooling is the mixed-model compromise: each group's estimate is pulled toward the overall mean, more so when the group has little data or lots of noise. This is shrinkage.

Why shrinkage is a good thing

Notice in the playground: when τ is large (groups really do differ a lot), there's little shrinkage: the data is trusted. When τ is small (groups are mostly alike), estimates collapse toward the overall mean. And at any setting, the small groups shrink more than the big ones, because they carry less independent information. You have already met this machinery under another name if you have read ahead: a random-effects meta-analysis is partial pooling with published studies as the groups, which is why a small study's estimate gets pulled toward the pooled effect.

The "G" in GLMM

Combine this grouping machinery with a link function and you get a generalized linear mixed model (GLMM): random effects on top of logistic or Poisson regression. That covers an enormous swath of real research: yes/no outcomes measured repeatedly within people, event counts nested within regions, and far more.

Problem 27 of the practice problems starts one step earlier, with a researcher deciding whether to run 90 people through one design each or 30 people through all three, which is the design choice that determines whether you need any of this machinery.