Non-Parametric Alternatives

When the assumptions behind a t-test or ANOVA don't hold (heavy skew, nasty outliers, tiny samples), you don't have to give up. Non-parametric tests throw away the raw numbers and work with their ranks instead. That one move makes them remarkably tough.

The trick: rank, don't measure

Most parametric tests care about how far apart values are. Rank-based tests only care about order: who sits above whom. Replace the numbers [2, 5, 9, 400] with their ranks [1, 2, 3, 4] and that monstrous 400 becomes just "the biggest one." Its ability to distort the result evaporates. That robustness to outliers and skew is the whole appeal.

The asymmetry is worth pausing on. As the highlighted value rockets upward, it inflates group 2's mean and its variance at the same time, yanking the t-statistic around. Sometimes the t-test even loses significance, because the variance blew up faster than the means pulled apart. The Mann-Whitney test barely reacts: the value was already the highest, so making it higher changes no ranks, and its p-value holds rock-steady.

The logic behind each test

Every rank-based test starts from the same instinct: swap the numbers for their order, then ask a question the order can answer. What changes from test to test is which order you build, and what you add up once you have it.

Mann–Whitney U: counting wins

Pool both groups, sort everyone into a single line, and hand out ranks from lowest to highest. The statistic U counts wins: across every pairing of a group-1 value against a group-2 value, how often does one side come out on top? That count falls straight out of the rank sums, because a group's total rank tells you how high its members sit in the shared order. Divide the winning count by the number of pairings and you get the probability of superiority: the chance that a value drawn from one group beats a value drawn from the other. A lone extreme value can only ever be "the largest rank," so pushing it further out changes nothing.

Wilcoxon signed-rank: magnitudes with a sign

For paired data, work with each person's difference score. Rank the absolute differences from smallest to largest, then re-attach the original signs. Adding up the ranks that carried a plus sign gives W₊ (the minus ranks give W₋, and the two are locked together). A plain sign test would only count how many people improved; the signed-rank test also weights how much each one moved, so a big change pulls harder than a barely-there one. The magnitudes enter only through their rank order, which is why the person with the wildest swing contributes the top rank whether they gained 13 points or 130.

Kruskal–Wallis: ANOVA on the ranks

Three or more independent groups reuse the same opening move: pool everyone, rank the whole pile, and total the ranks inside each group. If the groups were interchangeable, those rank totals should land roughly proportional to group size. The statistic H measures how far the observed totals stray from that even split, and under the null hypothesis it follows a chi-square distribution with k − 1 degrees of freedom. It is one-way ANOVA with ranks standing in for the raw scores.

Friedman: ranking within each person

Repeated measures rank within each person rather than across the whole sample. For every participant, rank their conditions from lowest to highest, so each row of the data turns into a small 1-2-3. Add those ranks down each column: if one condition genuinely tends to score highest, its column keeps collecting the top ranks. The statistic reads that imbalance and, like Kruskal–Wallis, compares it to a chi-square distribution with k − 1 degrees of freedom. Ranking inside each person is what cancels stable individual differences, the same trick the paired difference pulls for Wilcoxon.

Correlation has a rank-based version too. Spearman's ρ is just Pearson's r computed on the ranks, so it captures any steadily rising or falling relationship and shrugs off outliers. It lives alongside the other coefficients in the correlation lesson.

Which test replaces which

Each rank-based test steps in for a specific parametric one when its assumptions give way. The test chooser walks the same map from the other direction, starting from your design rather than from the test you were hoping to run:

The effect size your software prints

A rank test still owes you a size, not just a verdict. The parametric side has Cohen's d and η²; the rank side has the rank-biserial correlation, and the ledger above is already showing it to you under another name.

Recall the probability of superiority: the chance that a randomly picked value from one group beats a randomly picked value from the other. The rank-biserial correlation is that same quantity restretched onto the −1 to +1 scale a correlation lives on:

r_rb = 2 × P(superiority) − 1

Equivalently, it is the win rate minus the loss rate across every cross-group pairing. On the Mann–Whitney tab's starting data the ledger reports P(superiority) = .94, which puts r_rb at .89. Read it as a scoreboard: of the 36 possible pairings of one control value against one treatment value, treatment wins 34 and loses 2, and .89 is the gap between those two rates. Zero means the two groups interleave evenly; a magnitude of 1 means every value in one group beats every value in the other, and the sign records which group that is. (The ledger always reports the winning side's probability of superiority, so the formula above hands you the magnitude.)

Wilcoxon has a matched-pairs version of the same idea, (W₊ − W₋) / (W₊ + W₋), built from the signed-rank sums the ledger already displays.

Where this bites in practice: JASP prints the rank-biserial correlation when you tick Effect size in the analysis panel, and SPSS does not, which is why so many write-ups fall back on r = z/√N instead. Those two are different statistics, and on this same data they disagree: z = 2.48 over √12 gives r = .72 against the rank-biserial .89. Both are defensible; reporting which one you computed is not optional. If JASP is new to you, the thesis-data walkthrough takes a raw file through to a finished APA sentence.

The trade-off

Problem 18 of the practice problems runs a Mann–Whitney by hand on five dogs per shelter, from ranking the ten values through to the decision at α = .05.