Bayesian Thinking
Everything so far has been frequentist: probability means long-run frequency, and parameters are fixed-but-unknown numbers we estimate. Bayesian statistics reframes the whole thing. Probability becomes a degree of belief, parameters get full probability distributions, and learning from data is just one mechanical rule applied over and over: update your belief.
Bayes' rule, in words
Textbooks call it Bayes' rule or Bayes' theorem interchangeably; they are the same statement, and this lesson uses the shorter name. You start with a prior — what you believed before seeing the data. You collect data, whose likelihood says how well each possible parameter value explains what you saw. Multiply them and renormalize, and you get the posterior — your updated belief:
posterior ∝ likelihood × prior
That's the entire engine. The posterior from today becomes the prior for tomorrow, and beliefs evolve as evidence accumulates.
🎮 Prior → Data → Posterior
Estimating a proportion (say, a coin's chance of heads). Set your prior belief, then pour in data and watch the posterior (purple) form between your prior and what the data alone suggests.
What the playground shows
Data overwhelms the prior. With little data the posterior sits close to your prior. Slide the trials up and the posterior marches toward the observed rate and gets sharper — with enough evidence, your starting belief barely matters. Strong priors resist longer; flat (weak) priors let the data speak immediately.
- A flat / weak prior says "I don't know much" — the posterior is driven almost entirely by the data, and often looks a lot like the frequentist answer.
- A strong prior encodes real prior knowledge — useful when data is scarce, but it should be defensible, because it genuinely shifts your conclusions.
What "modern computing" actually does
The playground above draws an exact posterior because a proportion with a Beta prior is one of a small number of textbook cases where the multiplication has a closed-form answer. Realistic models almost never do, and for most of the twentieth century that was the practical objection to the whole framework: the rule was simple and the integral was impossible.
MCMC (Markov chain Monte Carlo) is what changed. Rather than solving for the posterior, the computer takes a long guided walk through parameter space, arranged so that it lingers in each region in proportion to how probable that region is. The trail it leaves behind is the posterior, delivered as a pile of samples you can average, plot, or read quantiles off, in the same spirit as bootstrapping replaces a formula with resampling. Stan, JAGS, PyMC and the Bayesian modules inside JASP all run some version of it, which is why a Bayesian analysis reports how many samples it drew and whether its chains agreed with each other.
Why people love (and argue about) Bayes
The Bayesian framework gives you exactly what you usually want: a full distribution for the parameter, so you can say "there's a 95% probability the rate is between X and Y" — a statement frequentist confidence intervals famously can't make. The flip side is the prior: it must be chosen, and different priors can give different answers when data is thin. Done transparently, that's a feature, not a bug.
Why it matters: Bayesian methods are everywhere now — A/B testing, clinical trials, machine learning, election forecasting. They handle uncertainty in a uniquely intuitive way, and modern computing has made them practical. Next we'll turn the posterior into concrete summaries with credible intervals.
Problem 31 in the practice problems runs the classic base-rate case by hand: a 95% sensitive test, a rare condition, and a positive result that leaves the patient at about 9%.
Common questions
What is the main difference between Bayesian and frequentist statistics?
What "probability" means. Frequentists treat parameters as fixed unknowns and put probability on data procedures ("5% of such intervals miss"); Bayesians put probability on parameter values themselves, as degrees of belief updated by evidence ("the rate is 95% likely between .55 and .72"). The Bayesian version answers the question people naturally ask — at the price of specifying a prior.
How strong is my prior, in units of data?
For a proportion there is an exact answer, and the playground above is built on it. A Beta(α, β) prior updates to Beta(α + successes, β + failures), so the prior enters the arithmetic as though you had already run α + β trials of which α succeeded. That total is precisely what the "prior confidence" slider sets. At its default of 6, twenty real trials outvote your belief better than three to one, and a posterior mean of .65 sits almost on top of an observed .70. Push it to 80 and the same twenty trials barely move anything. Two habits follow: state the prior in the paper as the number of pseudo-observations it is worth, and refit under a couple of defensible alternatives, because a conclusion that survives only one prior is a conclusion about the prior.
What is a Bayes factor?
The evidence ratio between two hypotheses: how much more probable the observed data is under H₁ than under H₀. BF = 10 means the data favors H₁ ten-to-one; BF = 1 means the data can't tell them apart. It's the Bayesian counterpart to significance testing, with two perks p-values lack: it can quantify evidence for a null, and it doesn't inflate with optional stopping.