Choosing Statistics for Your Dissertation

To pick the right analysis, answer three questions in order: what kind of variable is your outcome, what are you comparing or relating it to, and were the same people measured more than once? The answers point to one test. This guide goes through the questions, gives you the full map, and then covers the complications of real dissertations that flowcharts leave out.

The three questions

Every classical test answers a question of the form "Is [outcome] related to [structure], given [how the data were collected]?" So the choice breaks into three parts:

1. What kind of thing is the outcome? Continuous number, category, count, or time-until-something?
2. What is it being compared against or related to? Two groups, several groups, another continuous variable, several predictors?
3. Were the same people (or matched pairs) measured more than once? Paired data need paired tests.

To answer them interactively, use the Which test should I use? chooser, which asks the same questions in the same order. Plan My Analysis goes further and turns your answers into a printable plan with assumptions, a sample-size estimate, and an APA skeleton. This guide explains the reasoning, so you can understand your choice and defend it in your viva.

Question 1: What kind of thing is your outcome?

The outcome (dependent variable) is whatever your hypothesis says should change. Most of the choice depends on its data type:

• Continuous: reaction times, exam scores, blood pressure, questionnaire composites. Use the t-test, ANOVA or regression family.
• Categorical: passed or failed, chose A or B, diagnosed or not. Use chi-square (relating two categorical variables) or logistic regression (predicting a category).
• A count: number of errors, symptoms, visits. Small counts that cannot go below zero need a Poisson model, one of the generalized linear models.
• Time-until-an-event: days to relapse, weeks to dropout, where some people never have the event before the study ends. That censoring makes it survival analysis, not a t-test on the times.

Likert items come up in almost every dissertation. A single 5-point item is ordinal, so analyze it with rank-based methods. A composite of several items (a scale mean across 8 questions) is conventionally treated as continuous. That convention is defensible if you built the composite properly, including any reverse-coding (see Clean Your Survey Data), and its distribution is not severely non-normal. Say in your methods section which you did.

Question 2: What are you comparing or relating?

Now look at the other side of your hypothesis, the independent variable(s):

• Two groups → a t-test family member.
• Three or more groups → one-way ANOVA. Running several t-tests instead inflates your false-positive rate. ANOVA avoids that, and a significant result is followed by post-hoc tests, not more t-tests.
• Two grouping factors at once (say, therapy type × session format) → factorial ANOVA, which also tests the interaction: does the effect of one factor depend on the other?
• Another continuous variable → correlation if you just want the strength of association, simple regression if one variable predicts the other.
• Several predictors at once → multiple regression, the standard analysis in survey-based dissertations. Categorical predictors enter through dummy coding, and "does the effect of X depend on Z?" becomes an interaction term.

Question 3: Paired or independent?

If the same person contributes an observation to each condition (before/after, condition A and condition B, three time points), the observations are correlated. A test that assumes independence will then compute the wrong standard error. Paired designs need paired tests: the paired t-test for two measurements, repeated-measures ANOVA for more. The same applies to matched pairs (siblings, matched controls).

Pairing usually helps. Each person serves as their own control, so individual differences drop out of the error and power goes up: you need fewer participants to detect the same effect. The cost is carryover and order effects, and Between vs Within Designs shows when that cost outweighs the benefit. If your design has both a between-groups factor and repeated measurements, it is a mixed design, covered in the section on messy designs below.

The map

The table puts the three answers together. Each test links to its lesson, and the test in parentheses is the rank-based fallback for when the assumptions fail (see below).

"But I need to control for age…" (covariates)

Most dissertations adjust for something. The mechanics are simple. In a group design, adding a continuous covariate turns ANOVA into ANCOVA. In regression, you add the covariate as another predictor. There are two cautions. First, adjusting for a variable does not automatically remove bias. Adjusting for the wrong one (a mediator, or a collider) can create bias rather than remove it, and Causal DAGs & Confounding shows how to decide which variables belong in the model. Second, choose your covariates before you see the results. Adding covariates until the effect becomes significant is a common example of the garden of forking paths.

When reality is messy (it will be)

Real dissertations rarely match a row of the map exactly. The common cases:

• Unequal group sizes. Nothing in the t-test or ANOVA requires equal n. You lose a little power, and unequal variances distort the p-value more when the group sizes also differ, but unequal groups alone are not a problem.
• Assumptions failed. Do not switch tests because one normality check was significant. Look at the plots and the sample size first. ANOVA & t-Test Assumptions explains which violations matter. Unequal variances → Welch's version (one tick). Serious skew with small samples → the rank-based fallback in the map. Do not run all three and report the one you like best.
• Nested data. Students inside classrooms, patients inside clinics, trials inside participants: observations within a cluster resemble each other. That breaks independence and makes your p-values too small, with no warning in the output. Ignoring nesting is a common serious error in otherwise solid dissertations, and multilevel models are designed for this case.
• Mixed designs. A between factor plus repeated measures (treatment group × three time points) is a mixed ANOVA. It combines factorial and repeated-measures ANOVA, and it is increasingly analyzed as a multilevel model instead.
• Missing data. People skip questions and drop out. How you handle the gaps, and whether the missingness is random, can change your results considerably. Read Missing Data before you delete a single row.
• Your design fits nothing here. Describe the design precisely to your supervisor: outcome type, predictors, nesting, pairing. Do not force it into whichever test you happen to know. Most designs that do not fit this page still have a suitable analysis, and your supervisor or a statistician can help you find it.

A different question entirely: predicting, not testing

Everything above tests hypotheses: is there an effect, how big is it, and how uncertain is the estimate? Your question may instead be "how accurately can I predict who drops out?", where what matters is accuracy on new cases by whatever method works. That is machine learning, and there a model is judged by its accuracy on held-out data, not by p-values. Prediction vs Explanation explains the difference, and the ML course covers the methods. The same regression can serve either goal, judged differently for each.

Decide before you collect

Choose your test before you have data. Doing so forces your hypothesis to be precise. It lets you compute how many participants you need, which the power calculator does for you. (A study with too few participants can miss a real effect and end at p = .12.) And a written plan protects you from the temptation to try analyses until one is significant. Preregistration makes the plan official. Plan My Analysis drafts it (test, assumptions, sample size, APA skeleton) on one printable page your supervisor can approve in a meeting.

Keep going

• Which test should I use? this guide as a 60-second interactive.
• Plan My Analysis: turn your answers into a printable analysis plan.
• Hypothesis-Testing Logic: what a test asks, before you pick one.
• From Question to Hypothesis: if the research question itself is still fuzzy.
• Analyze Your Thesis Data in JASP: running the test you chose.
• From SPSS Output to APA Results: reading the output and reporting it correctly.
• One Study, Start to Finish: one project that makes each of these choices step by step.