Guide

Choosing Statistics for Your Dissertation

There is no test-choosing talent, and you don't need to memorize a flowchart. Picking the right analysis is three boring questions asked in the right order: what kind of thing is your outcome, what structure are you comparing or relating, and were the same people measured more than once? Answer those and the test names itself. This guide walks the questions slowly, gives you the full map, and then says the honest things about real dissertations that flowcharts leave out.

The three questions

Every classical test is an answer to a sentence of the form: "Is [outcome] related to [structure], given [how the data were collected]?" So the choice decomposes:

  1. What kind of thing is the outcome? Continuous number, category, count, or time-until-something?
  2. What is it being compared against or related to? Two groups, several groups, another continuous variable, several predictors?
  3. Were the same people (or matched pairs) measured more than once? Paired data need paired tests.

If you'd rather answer these interactively, the Which test should I use? chooser asks exactly this sequence, and Plan My Analysis goes further, turning your answers into a printable plan with assumptions, a sample-size estimate, and an APA skeleton. This guide is the version you read to understand the choice, so you can defend it in your viva.

Question 1: What kind of thing is your outcome?

The outcome (dependent variable) is whatever your hypothesis says should change. Its data type does most of the deciding:

  • Continuous: reaction times, exam scores, blood pressure, questionnaire composites. Opens the t-test/ANOVA/regression family.
  • Categorical: passed or failed, chose A or B, diagnosed or not. Points to chi-square (relating two categoricals) or logistic regression (predicting a category).
  • A count: number of errors, symptoms, visits. Small, bounded-at-zero counts are Poisson territory: generalized linear models.
  • Time-until-an-event: days to relapse, weeks to dropout, where some people never have the event before the study ends. That censoring makes it survival analysis, not a t-test on the times.

The Likert question, because every dissertation asks it: a single 5-point item is ordinal — treat it with rank-based methods. A composite of several items (a scale mean across 8 questions) is conventionally treated as continuous, and that convention is defensible, provided you built the composite properly (reverse-coding and all: see Clean Your Survey Data) and its distribution isn't pathological. Say in your methods section which you did.

Question 2: What are you comparing or relating?

Now look at the other side of your hypothesis, the independent variable(s):

  • Two groups → a t-test family member.
  • Three or more groups → one-way ANOVA (running several t-tests instead inflates your false-positive rate, which is the whole reason ANOVA exists, and why significant results are followed by post-hoc tests, not more t-tests).
  • Two grouping factors at once (say, therapy type × session format) → factorial ANOVA, which also buys you the interaction: does the effect of one factor depend on the other? Interactions are often the most publishable thing in a dissertation.
  • Another continuous variable → correlation if you just want the strength of association, simple regression if one variable predicts the other.
  • Several predictors at once → multiple regression, the workhorse of survey-based dissertations. Categorical predictors join via dummy coding; "does the effect of X depend on Z?" becomes an interaction term.

Question 3: Paired or independent?

The quiet question that changes everything. If the same person contributes an observation to each condition — before/after, condition A and condition B, three time points — the observations are correlated, and a test that assumes independence will get its arithmetic wrong. Paired designs get paired tests: the paired t-test for two measurements, repeated-measures ANOVA for more. The same applies to matched pairs (siblings, matched controls).

Pairing is usually a gift — each person serves as their own control, which strips out individual differences and buys power (fewer participants for the same detection ability). The price is carryover and order effects; Between vs Within Designs shows when the gift turns poisonous. And if your design mixes both (a between-groups factor and repeated measurements), that's a mixed design, which brings us to the messy section below.

The map

The three answers, laid out. Every test links to its interactive lesson; the parenthetical is the rank-based fallback for when assumptions crumble (more on that below).

Your situationThe testLearn it
Continuous outcome, 2 independent groupsIndependent t (Mann–Whitney)Stats 1.12
Continuous outcome, same people twicePaired t (Wilcoxon)Stats 1.11
Continuous outcome, 3+ independent groupsOne-way ANOVA (Kruskal–Wallis)Stats 2.1
Continuous outcome, same people 3+ timesRM-ANOVA (Friedman)Stats 2.4
Continuous outcome, two crossed factorsFactorial ANOVAStats 2.3
Two continuous variables, associationPearson r (Spearman ρ)Stats 2.8
Continuous outcome, one continuous predictorSimple regressionStats 2.9
Continuous outcome, several predictorsMultiple regressionStats 3.1
Two categorical variablesChi-square (Fisher's exact for tiny cells)Stats 2.7
Categorical (yes/no) outcome, any predictorsLogistic regressionStats 3.7
Group comparison, adjusting for a covariateANCOVAStats 3.4
Count outcomePoisson / negative binomial GLMStats 4.4
Time-to-event outcome (with censoring)Survival analysisStats 4.8
People nested in groups (classes, clinics)Multilevel modelsStats 4.5

"But I need to control for age…" (covariates)

Almost every dissertation wants to adjust for something. Mechanically it's easy: in a group design, adding a continuous covariate turns ANOVA into ANCOVA; in regression, you simply add the covariate as another predictor. Two honest cautions. First, "controlling for" is not a purification ritual; adjusting for the wrong variable (a mediator, or a collider) can create bias rather than remove it, and Causal DAGs & Confounding is the map for which variables belong in the model. Second, decide your covariates before seeing the results. Covariates added until the effect becomes significant is one of the garden of forking paths' best-trodden trails.

When reality is messy (it will be)

Real dissertations rarely match a textbook row exactly. The honest answers:

  • Unequal group sizes. Fine. Nothing in the t-test or ANOVA requires equal n; you lose a little power and equal-variance violations bite harder, but unequal groups alone are not a problem.
  • Assumptions failed. Don't panic-switch tests over one significant normality check; look at the plots and the sample size first, because Assumptions & When They Break triages which violations actually matter. Unequal variances → Welch's version (one tick). Serious skew with small samples → the rank-based fallback in the map. What's not allowed: trying all three and reporting the one you liked best.
  • Nested data. Students inside classrooms, patients inside clinics, trials inside participants: observations within a cluster resemble each other, which breaks independence and silently shrinks your p-values. Ignoring it is the most common serious error in otherwise-solid dissertations, and multilevel models exist for exactly this.
  • Mixed designs. A between factor plus repeated measures (treatment group × three time points) is a mixed ANOVA, conceptually just the marriage of factorial and repeated-measures ANOVA, and increasingly analyzed as a multilevel model instead.
  • Missing data. People skip questions and drop out. How you handle it (and whether the missingness is random) changes your answer more than you'd like; consult Missing Data before you delete a single row.
  • Your design fits nothing here. That's a finding, not a failure. Describe the design precisely — outcome type, predictors, nesting, pairing — to your supervisor, and resist forcing it into whichever test you happen to know. The right analysis for a design that doesn't fit this page exists; it's just a conversation, not a guess.

A different question entirely: predicting, not testing

Everything above tests hypotheses: is there an effect, how big, with what uncertainty. If your question is instead "how accurately can I predict who drops out?" — accuracy on new cases, whatever works — you've crossed into machine learning, where the rules change (evaluation on held-out data replaces p-values). Prediction vs Explanation marks the border, and the ML course takes it from there. The same regression can serve either goal; what changes is how you judge it.

Decide before you collect

The best time to choose your test is before you have data: it forces your hypothesis to be precise, it lets you compute how many participants you need (the power calculator answers that exactly; underpowered studies are how promising dissertations end in p = .12), and a written-down plan is armor against the temptation to shop for significance later. Preregistration makes the plan official; Plan My Analysis will draft it (test, assumptions, sample size, APA skeleton) on one printable page your supervisor can sign off in a meeting.

The whole guide in one sentence: name your outcome's type, name the structure it's related to, say whether measurements repeat, then read the map, check the assumptions, and commit in writing before the data arrive.

Keep going