Tools

Practice Problems

The quiz and the flashcards ask you to recognize a right answer. This page asks you to produce one. Every problem here gives you numbers small enough for a calculator and a sheet of paper, and every solution shows the whole path: the formula, the numbers going into it, the arithmetic, and the sentence you would actually write in a report.

How to use it: read the scenario, work the answer out on paper, and open the solution only afterwards. The value is in comparing your working to the steps, not your final number to an answer key. If a step loses you, the § link under each solution goes straight to the lesson it comes from.

For teachers: the first button suppresses every solution, whether or not it is open on screen.

📊 Stats 1 · Foundations

Twenty-five problems tracking the Stats 1 syllabus: what kind of data you have, how to describe it, how far a score sits from average, how a sample mean turns into a confidence interval and then a z-test or a t-test, the probability rules that three of them run on, then a proportion and a two-way table of counts, and finally a correlation and a fitted line worked by hand.

1What the library actually measured

Warm-up

A city library logs eight things about every visitor who fills in the front-desk survey. The systems team wants a monthly report and has asked for "the average of each column".

The eight columns
  • Membership number (e.g. 40118)
  • Age in whole years
  • Postcode district (M14, M20, M21…)
  • Satisfaction, 1 = very dissatisfied to 5 = very satisfied
  • Books borrowed in the last year
  • Reading-room temperature in °C at the time of the visit
  • Branch (Central, Didsbury, Withington)
  • Position in the returns queue (1st, 2nd, 3rd…)
  1. Give the level of measurement for each column.
  2. Which columns can be averaged without an argument?
  3. The report comes back with a mean postcode district of 17.4. What has gone wrong?
  4. Is a reported mean satisfaction of 3.8 defensible?
Worked solution
  1. Classify each column
    ColumnLevelWhy
    Membership numberNominalNumeric, but it is a name for a person, not a quantity
    Age in yearsRatioEqual intervals and a real zero, so 40 is twice 20
    Postcode districtNominalA label with digits in it
    Satisfaction 1–5OrdinalOrdered, but the gap 1→2 need not equal 4→5
    Books borrowedRatioA count; zero books means zero borrowing
    Temperature °CIntervalEqual intervals, arbitrary zero: 20 °C is not twice as warm as 10 °C
    BranchNominalUnordered categories
    Queue positionOrdinalRanks; the gap between 1st and 2nd is not a fixed quantity
  2. Which can be averaged Age, books borrowed and temperature. Interval and ratio data support addition and division, which is all a mean is.
  3. The postcode mean The digits in M14 encode a place, not an amount. Averaging them produces a number that refers to nothing: there is no location "17.4", and moving from M14 to M20 is not "six units" of anything.
  4. Mean satisfaction of 3.8 Strictly, a mean assumes equal spacing, and Likert points are only ordered. In practice psychology and market research report Likert means constantly, and with five or more points and roughly symmetric responses the distortion is small. Report the median and the full response distribution alongside it, and the reader can judge for themselves.

Three of the eight columns are nominal, two are ranks, one is interval and two are ratio. Only age, books borrowed and temperature are safely averageable, and satisfaction is the one worth a footnote.

From §1.2 Types of Data.

2The bakery's odd Saturday

Warm-up

A small bakery counts the sourdough loaves it sells each day for nine days. One of those days was the street festival.

Loaves sold, nine days 12, 14, 15, 16, 17, 19, 21, 25, 41
  1. Compute the mean and the median.
  2. Compute the sample standard deviation.
  3. Which of the two centers should the baker use to plan next Tuesday's flour order, and why?
Worked solution
  1. Mean x̄ = Σx / n Σx = 12+14+15+16+17+19+21+25+41 = 180 x̄ = 180 / 9 = 20.0 loaves
  2. Median The values are already in order and n = 9 is odd, so the median is the 5th value. Median = 17 loaves
  3. Deviations and their squares (x − x̄): −8, −6, −5, −4, −3, −1, 1, 5, 21 (x − x̄)²: 64, 36, 25, 16, 9, 1, 1, 25, 441 Σ(x − x̄)² = 618
  4. Variance, then standard deviation s² = Σ(x − x̄)² / (n − 1) = 618 / 8 = 77.25 s = √77.25 = 8.789 ≈ 8.79 loaves Divide by n − 1, not n: these nine days are a sample of the bakery's trading, not the whole of it.
  5. Which center to plan with The mean of 20 is dragged up by the single festival day. Drop that day and the mean of the remaining eight is 139 / 8 = 17.4, which lands right next to the median. For an ordinary Tuesday the median of 17 is the honest planning figure, and the festival day belongs in the plan as its own event.

Mean = 20.0, median = 17, s = 8.79 loaves. Order for about 17, and order extra for festival days separately.

Rounding: keep the full 8.7892 if the figure feeds another calculation, and round to 8.79 only when reporting.

From §1.3 Describing Data · check it in the Descriptives calculator.

3Priya's two marks

Warm-up

Priya sat two papers that were marked on different scales by different examiners. She wants to know which one she actually did better on.

Given
  • Statistics: Priya scored 68. Cohort M = 60, SD = 10.
  • Research Methods: Priya scored 74. Cohort M = 65, SD = 12.
  1. Convert both marks to z-scores.
  2. Which paper was her stronger performance relative to the cohort?
  3. Express both as T-scores, where T = 50 + 10z.
Worked solution
  1. Formula z = (x − μ) / σ
  2. Statistics z = (68 − 60) / 10 = 8 / 10 = 0.80
  3. Research Methods z = (74 − 65) / 12 = 9 / 12 = 0.75
  4. Compare The raw marks say Methods (74 > 68). The z-scores say Statistics, by a whisker: 0.80 SDs above her cohort against 0.75. The bigger raw mark came on the paper where everyone scored higher and the spread was wider.
  5. T-scores T = 50 + 10z Statistics: 50 + 10(0.80) = 58.0 Methods: 50 + 10(0.75) = 57.5

Statistics z = 0.80 (T = 58.0), Methods z = 0.75 (T = 57.5). Her Statistics paper was marginally the stronger performance, and the raw marks pointed the other way.

From §1.5 Z-Scores & The Normal Distribution.

4Resting heart rate at the clinic

Exam-level

A clinic's records show that resting heart rate in healthy adults is close to normally distributed with a mean of 72 bpm and a standard deviation of 8 bpm. The nurse wants three numbers for a screening protocol.

Given μ = 72 bpm, σ = 8 bpm, distribution approximately normal
  1. What proportion of adults have a resting heart rate below 60 bpm?
  2. What proportion fall between 64 and 84 bpm?
  3. Above what rate do the highest 5% of adults sit? Round to a whole bpm.
Worked solution
  1. Below 60 bpm: standardize z = (60 − 72) / 8 = −12 / 8 = −1.50 P(Z < −1.50) = .0668 About 6.7% of adults, or roughly 1 in 15.
  2. Between 64 and 84 bpm: two z-scores z₁ = (64 − 72) / 8 = −1.00 z₂ = (84 − 72) / 8 = +1.50
  3. Subtract the two cumulative areas P(−1.00 < Z < 1.50) = P(Z < 1.50) − P(Z < −1.00) = .9332 − .1587 = .7745 About 77.5%.
  4. The top 5%: go backwards from the area The cutoff leaves .95 below it, so z = 1.645 x = μ + zσ = 72 + 1.645(8) = 72 + 13.16 = 85.16 Round to 85 bpm for the protocol.

6.7% below 60 bpm, 77.5% between 64 and 84 bpm, and the top 5% sit above roughly 85 bpm.

Rounding: 1.645 is the conventional two-decimal z for the 95th percentile (1.6449 exactly). Using 1.64 gives 85.12 and using 1.65 gives 85.20, so at whole-bpm precision the choice does not matter here. It would matter if the protocol were written to one decimal.

From §1.5 Z-Scores & The Normal Distribution · check the areas in Statistical Tables.

5Who cycles to work

Warm-up

A company surveys all 200 of its staff about commuting and cross-tabulates the answers by age band.

Counts
CyclesDoes notTotal
Under 304852100
30 or over2278100
Total70130200
  1. Pick one employee at random. What is P(cycles)?
  2. What is P(cycles | under 30)?
  3. What is P(under 30 and cycles)?
  4. Are age band and cycling independent?
Worked solution
  1. Marginal probability P(cycles) = 70 / 200 = .35
  2. Conditional probability: the denominator shrinks P(cycles | under 30) = 48 / 100 = .48 Conditioning on "under 30" throws away the other row, so the denominator is that row's total, not 200.
  3. Joint probability P(under 30 and cycles) = 48 / 200 = .24
  4. Independence check If independent: P(A and B) = P(A) × P(B) P(under 30) × P(cycles) = .50 × .35 = .175 Observed joint = .24 ≠ .175
  5. Read it the other way too P(cycles | under 30) = .48 against P(cycles) = .35. Knowing someone's age band changes the probability, which is exactly what dependence means.

.35, .48, .24, and no: the two variables are dependent. Under-30s cycle at nearly twice the rate of over-30s (48% against 22%).

From §1.7 Probability Basics · test the dependence formally with a chi-square test (Problem 19 below).

6Sharpening the estimate

Exam-level

A driving-simulator lab knows from years of testing that braking reaction times in its population have a standard deviation of 60 ms. A new study will estimate the population mean.

Given σ = 60 ms (population SD, known)
  1. What is the standard error of the mean for a sample of 25 drivers?
  2. The lab raises the sample to 100. What happens to the standard error?
  3. How many drivers would be needed for a standard error of 4 ms?
  4. In one sentence, what does the standard error describe that the SD does not?
Worked solution
  1. Formula SE = σ / √n
  2. n = 25 SE = 60 / √25 = 60 / 5 = 12 ms
  3. n = 100 SE = 60 / √100 = 60 / 10 = 6 ms Four times the sample halves the standard error, because √4 = 2. Precision is bought with the square root, which is why the second hundred participants buy far less than the first hundred.
  4. Solve for n 4 = 60 / √n → √n = 60 / 4 = 15 n = 15² = 225 drivers
  5. SD against SE The SD describes how much individual drivers differ from each other, and it does not shrink when you collect more of them. The SE describes how much a sample mean would bounce around from study to study, and it shrinks as √n grows.

SE = 12 ms at n = 25, 6 ms at n = 100, and 225 drivers are needed for an SE of 4 ms.

From §1.9 Sampling Distributions.

7Thirty-six pizzas

Exam-level

A pizza chain's delivery times have a mean of 28 minutes and a standard deviation of 9 minutes. The distribution is clearly right-skewed: most deliveries cluster near the mean, and a long tail of bad nights stretches out to over an hour. A regional manager samples 36 deliveries.

Given μ = 28 min, σ = 9 min, right-skewed population, n = 36
  1. The population is skewed. Why can the manager still use the normal curve for the sample mean?
  2. What is the probability the sample mean exceeds 30 minutes?
  3. What is the probability it falls between 27 and 29 minutes?
Worked solution
  1. Why normality applies here The central limit theorem is about the distribution of the sample mean, not about the raw deliveries. With n = 36 the sampling distribution of x̄ is close to normal even though individual delivery times are not. The skew has to be extreme, or n small, for this to fail.
  2. Standard error SE = σ / √n = 9 / √36 = 9 / 6 = 1.5 min
  3. P(x̄ > 30) — standardize the mean, not a delivery z = (30 − 28) / 1.5 = 2 / 1.5 = 1.333 P(Z > 1.333) = 1 − .9088 = .0912
  4. P(27 < x̄ < 29) z₁ = (27 − 28) / 1.5 = −0.667 z₂ = (29 − 28) / 1.5 = +0.667 P(−0.667 < Z < 0.667) = .7475 − .2525 = .4950
  5. Sanity check A single delivery taking over 30 minutes is common. A mean of 36 deliveries doing so is not, at about 9%. Averaging is what kills the variability.

The CLT covers the sample mean regardless of the skew. P(x̄ > 30) = .091, and P(27 < x̄ < 29) = .495, so almost exactly half of such samples land within a minute of the true mean.

Rounding: carry z = 1.333, not 1.3. Rounding to 1.3 gives .0968 instead of .0912, an error of more than half a percentage point.

From §1.10 Central Limit Theorem.

8How much do first-years sleep?

Exam-level

A student union surveys 25 randomly selected first-years about weeknight sleep. The sample averages 6.8 hours with a standard deviation of 1.5 hours. Sleep hours are roughly symmetric with no outliers.

Given n = 25, x̄ = 6.8 h, s = 1.5 h
  1. Build a 95% confidence interval for the mean weeknight sleep of first-years.
  2. Public health guidance recommends 8 hours. What does the interval say about that?
  3. The union repeats the survey with 100 students and gets the same mean and SD. How does the interval change?
Worked solution
  1. Formula: t, not z, because σ is unknown CI = x̄ ± t* × (s / √n), df = n − 1
  2. Standard error SE = 1.5 / √25 = 1.5 / 5 = 0.30 h
  3. Critical value df = 25 − 1 = 24, t*(.025, 24) = 2.064
  4. Margin of error 2.064 × 0.30 = 0.619 h
  5. The interval 6.8 ± 0.619 = [6.18, 7.42] hours
  6. Against the 8-hour guidance 8 sits well outside the interval, so 8 hours is not a plausible value for the population mean given this sample. Equivalently, a one-sample t-test against 8 would reject at α = .05. What the interval does not say is that 95% of first-years sleep between 6.18 and 7.42 hours: the interval is about the mean, not about individuals.
  7. With n = 100 SE = 1.5 / 10 = 0.15, df = 99, t* = 1.984 margin = 1.984 × 0.15 = 0.298 CI = [6.50, 7.10] hours Four times the sample gives an interval about half as wide, and t* also drifts down towards 1.96 as df grows.

95% CI [6.18, 7.42] hours. First-years sleep less than the recommended 8 hours, and quadrupling the sample narrows the interval to [6.50, 7.10].

From §1.11 Confidence Intervals · look up t* in Statistical Tables.

9What Marek thinks p means

Exam-level

An allotment society tests a seaweed feed on tomatoes. Twenty-four plots, twelve fed and twelve not, yield measured in kilograms. The write-up reports a mean difference of 0.42 kg, t(22) = 2.31, p = .030. Marek reads it and writes in the newsletter: "There is only a 3% chance the seaweed feed makes no difference."

  1. State what p = .030 actually means here.
  2. Marek's neighbor replies: "So there is a 97% chance the effect is real." Is that a different mistake?
  3. At α = .05, what is the decision, and which error could it be?
  4. The society repeats the trial on 8 plots and gets p = .21. Has it now shown the feed does nothing?
  5. If twenty societies each ran an honest trial of a feed that truly does nothing, at α = .05, how many "significant" results would you expect to see published?
Worked solution
  1. What p actually is If the feed truly made no difference, 3% of trials run this way would produce a difference of at least 0.42 kg in one direction or the other. It is the probability of data this extreme given the null hypothesis, written P(data | H₀).
  2. Marek's error He has flipped the conditional into P(H₀ | data). Those two are not interchangeable, and turning one into the other needs a prior probability that the test never supplies.
  3. The neighbor's version Same inversion, wearing a hat. 1 − p is not the probability that the alternative is true, and it is not the probability the result replicates either.
  4. The decision p = .030 < α = .05 → reject H₀ If the feed genuinely does nothing, this rejection is a Type I error. Setting α = .05 caps that error rate at 5% over the long run; it says nothing about whether this particular result is the mistake.
  5. The 8-plot repeat No. A non-significant result means the data are compatible with no effect, not that the effect is zero. With 8 plots the test has very little power, so a real effect of this size would be missed most of the time. Failing to detect it is a Type II error. Report the confidence interval: if it stretches from −0.3 to +1.1 kg, the study is simply too small to settle the question.
  6. Twenty null trials Expected false positives = 20 × .05 = 1 One society in twenty would find "significant" evidence for a feed that does nothing. If only the exciting result gets into the newsletter, the newsletter reports an effect that is not there.

p is P(data at least this extreme | H₀ true), so both Marek and his neighbor have inverted the conditional. The decision is to reject at α = .05, with Type I error as the risk. The 8-plot follow-up shows nothing either way, and one false positive per twenty null trials is the expected background rate.

From §1.12 Hypothesis Testing Logic and §3.1 Effect Size & Power.

10Does the typing course deliver?

Exam-level

A typing course advertises that its graduates average 65 words per minute. A consumer group tests 16 graduates chosen at random and records a mean of 61.5 wpm with a standard deviation of 6.4 wpm.

Given μ₀ = 65 wpm (the claim), n = 16, x̄ = 61.5, s = 6.4, α = .05, two-tailed
  1. State the hypotheses.
  2. Compute the test statistic and its degrees of freedom.
  3. Make the decision against the critical value, and report the p-value.
  4. Compute Cohen's d and write the result as you would in a report.
Worked solution
  1. Hypotheses H₀: μ = 65 H₁: μ ≠ 65 Two-tailed, because the consumer group would report the course being better than claimed as well as worse.
  2. Standard error SE = s / √n = 6.4 / √16 = 6.4 / 4 = 1.60
  3. Test statistic t = (x̄ − μ₀) / SE = (61.5 − 65) / 1.60 t = −3.5 / 1.60 = −2.1875 df = n − 1 = 15
  4. Critical value t*(.025, 15) = 2.131 |−2.1875| = 2.188 > 2.131 → reject H₀
  5. p-value p = 2 × P(T₁₅ > 2.1875) = .045
  6. Effect size d = (x̄ − μ₀) / s = −3.5 / 6.4 = −0.55 A medium shortfall of just over half a standard deviation.
  7. Write-up Graduates typed more slowly than the advertised 65 wpm, M = 61.5, SD = 6.4, t(15) = −2.19, p = .045, d = −0.55, 95% CI [58.09, 64.91].

Reject H₀. The 65 wpm claim is not supported, though at p = .045 with n = 16 the evidence is thin, and the confidence interval reaches almost up to 65.

Rounding matters here. Carry t = −2.1875 rather than rounding the standard error first; this test clears its critical value by 0.06, so an early round-off can flip the decision. Report t to two decimals and p to three.

From §1.13 One-Sample & Paired t-Tests · format the sentence with the APA formatter.

11Ten students, one mnemonic

Exam-level

Ten students learn a mnemonic technique. Each recalls a 20-item word list before training and a matched list afterwards. The difference scores (after − before) are recorded.

Difference scores (after − before), n = 10 3, 5, −1, 4, 6, 2, 0, 7, 3, 1
  1. Why is a paired test right here rather than an independent-samples test?
  2. Compute the mean difference and its standard deviation.
  3. Run the paired t-test at α = .05, two-tailed.
  4. Report the effect size.
Worked solution
  1. Why paired Each after-score is tied to a specific person's before-score. Subtracting within a person removes that person's baseline memory ability from the comparison, which is where most of the noise lives. An independent test would treat the twenty scores as twenty unrelated people and throw that advantage away.
  2. Mean difference Σd = 3+5−1+4+6+2+0+7+3+1 = 30 d̄ = 30 / 10 = 3.00 items
  3. SD of the differences (d − d̄): 0, 2, −4, 1, 3, −1, −3, 4, 0, −2 (d − d̄)²: 0, 4, 16, 1, 9, 1, 9, 16, 0, 4 → Σ = 60 s_d = √(60 / 9) = √6.667 = 2.582
  4. Standard error and t SE = s_d / √n = 2.582 / √10 = 2.582 / 3.162 = 0.8165 t = d̄ / SE = 3.00 / 0.8165 = 3.674 df = n − 1 = 9
  5. Decision t*(.025, 9) = 2.262 3.674 > 2.262 → reject H₀ p = 2 × P(T₉ > 3.674) = .005
  6. Effect size for a paired design d_z = d̄ / s_d = 3.00 / 2.582 = 1.16 Large. Note that d_z is scaled by the SD of the differences, so it is not comparable with a between-groups d computed from raw score SDs. Say which one you used.

Recall improved by 3.00 items on average, t(9) = 3.67, p = .005, d_z = 1.16. With ten students and no control group, this shows the scores went up; it does not rule out practice or test-order effects.

From §1.13 One-Sample & Paired t-Tests · effect-size conversions in the converter.

12The study that could not see straight

Stretch

Two ways of teaching fractions are compared in a small trial. Twelve children learn by manipulatives, twelve by worked examples, and all twenty-four sit the same end-of-topic test out of 100.

Given
  • Manipulatives: n = 12, M = 78.0, s = 8.0
  • Worked examples: n = 12, M = 71.5, s = 9.0
  • α = .05, two-tailed; variances judged similar enough to pool
  1. Run the independent-samples t-test.
  2. Compute Cohen's d.
  3. The researcher concludes "the two methods work equally well". Assess that.
  4. How many children per group would an 80%-powered replication need?
Worked solution
  1. Pooled variance s²_p = [(n₁−1)s₁² + (n₂−1)s₂²] / (n₁ + n₂ − 2) = [11(64) + 11(81)] / 22 = (704 + 891) / 22 = 1595 / 22 = 72.50 → s_p = √72.50 = 8.515
  2. Standard error of the difference SE = s_p × √(1/n₁ + 1/n₂) = 8.515 × √(1/12 + 1/12) = 8.515 × √0.1667 = 8.515 × 0.4082 = 3.476
  3. Test statistic t = (78.0 − 71.5) / 3.476 = 6.5 / 3.476 = 1.870 df = 12 + 12 − 2 = 22
  4. Decision t*(.025, 22) = 2.074 1.870 < 2.074 → fail to reject H₀, p = .075
  5. Effect size d = (M₁ − M₂) / s_p = 6.5 / 8.515 = 0.76 By Cohen's benchmarks that is medium-to-large, just short of the 0.80 mark, sitting inside a non-significant test.
  6. Assess the conclusion "Equally well" is not supported. A non-significant result means the data do not rule out zero difference, and here the best estimate of the difference is 6.5 marks with d = 0.76. With 12 per group the study had about 43% power to detect an effect that size, so it was more likely to miss it than to find it. The honest sentence is that the trial was too small to decide.
  7. Sample size for a replication Two-sample t, d = 0.76, α = .05, power = .80 n = 29 per group (58 children total) 28 per group falls just short at 79.8% power, which is why the answer rounds up rather than down. Planning for a medium d = 0.50 instead would need 64 per group. Powering a study on the effect size your own underpowered pilot produced is optimistic, because a pilot that reached significance has almost certainly overestimated the effect.

t(22) = 1.87, p = .075, d = 0.76. Not significant, not "no difference": a large point estimate measured with far too little precision. A replication needs at least 29 per group.

From §1.14 Independent Samples t-Test and §3.1 Effect Size & Power · redo the sample size in the Power calculator.

13Two electives, one overlap

Warm-up

A department checks the records of all 400 final-year students. Two electives are on offer, and a student may take either, both, or neither.

Given
  • Took the statistics elective: 240
  • Took the research-methods elective: 180
  • Took both: 120
  1. Pick one student at random. What is P(statistics or methods)?
  2. What is P(neither elective)?
  3. How many took statistics but not methods?
  4. Are the two electives independent?
Worked solution
  1. The three probabilities you are handed P(S) = 240 / 400 = .60 P(M) = 180 / 400 = .45 P(S and M) = 120 / 400 = .30
  2. General addition rule P(S or M) = P(S) + P(M) − P(S and M) = .60 + .45 − .30 = .75 Adding .60 and .45 gives 1.05, which is already impossible. The 120 students in the overlap were counted inside each total, so exactly one copy of them has to come back out.
  3. Check it by counting people 240 + 180 − 120 = 300 students → 300 / 400 = .75
  4. Neither, by the complement rule P(neither) = 1 − P(S or M) = 1 − .75 = .25 → 100 students
  5. Statistics but not methods 240 − 120 = 120 students → .30 Methods but not statistics is 180 − 120 = 60, and the four groups 120 + 120 + 60 + 100 rebuild the 400.
  6. Independence check If independent: P(S and M) = P(S) × P(M) = .60 × .45 = .27 Observed: .30 ≠ .27 P(M | S) = 120 / 240 = .50 against P(M) = .45 Not independent, though the dependence is mild. Taking statistics lifts the chance of taking methods from .45 to .50, which is a real association and a small one.

.75, .25, 120 students, and no: .30 against the .27 independence would require. The gap is small, so a formal test would be worth running before making anything of it.

From §1.7 Probability Basics · the same table shape is tested for independence in §1.16 Chi-Square Tests.

14The screening letter

Exam-level

A health service offers a screening test for a condition that 2% of the eligible population has. The test detects 95% of the people who have it, and correctly clears 90% of the people who do not. Ten thousand people are screened. One of them opens a letter saying the result was positive.

Given
  • Prevalence: P(ill) = .02
  • Sensitivity: P(positive | ill) = .95
  • Specificity: P(negative | healthy) = .90
  • Cohort: 10,000 people
  1. Fill in the four leaf counts of the tree.
  2. What is P(ill | positive)?
  3. What is P(healthy | negative)?
  4. The leaflet calls the test "95% accurate". What share of all 10,000 results does it actually get right, and is that the number the letter's reader needs?
Worked solution
  1. Split the cohort by the condition Ill: 10,000 × .02 = 200 Healthy: 10,000 − 200 = 9,800
  2. Split each branch by the test result Ill and positive: 200 × .95 = 190 Ill and negative: 200 − 190 = 10 Healthy and positive: 9,800 × .10 = 980 Healthy and negative: 9,800 − 980 = 8,820 The four leaves add back to 10,000, which is the arithmetic check worth doing every time.
  3. P(ill | positive): condition on the positives Positives in total: 190 + 980 = 1,170 P(ill | positive) = 190 / 1,170 = .162 → 16.2% The same answer from Bayes' rule, with the law of total probability underneath the line: (.02 × .95) / (.02 × .95 + .98 × .10) = .019 / .117 = .162.
  4. P(healthy | negative): condition on the negatives Negatives in total: 10 + 8,820 = 8,830 P(healthy | negative) = 8,820 / 8,830 = .9989 → 99.9% A negative result here is close to conclusive, while a positive one is mostly a false alarm. The same test does two very different jobs depending on which way it points.
  5. What "accurate" means, and what it hides Correct results: 190 + 8,820 = 9,010 → 9,010 / 10,000 = 90.1% The 95% in the leaflet is the sensitivity, P(positive | ill), which describes the test's behavior on people already known to be ill. The reader of the letter is in the opposite position and needs P(ill | positive), which is 16.2%. Overall accuracy, 90.1% here, is a third number again, and it is flattered by the 8,820 healthy people the test clears without effort.

190 / 10 / 980 / 8,820. P(ill | positive) = 16.2%, P(healthy | negative) = 99.9%, overall accuracy 90.1%. Roughly five out of six positive letters go to someone who does not have the condition, which is why a positive screen is normally followed by a second, more specific test rather than a diagnosis.

From §1.7 Probability Basics · rebuild the tree with the sliders in that lesson's Base-Rate Tree.

15Bikes back at the dock

Exam-level

A bike-share operator records X, the number of bikes returned to one quiet dock in an hour, over a long stretch of weekday mornings. The relative frequencies settle down into the distribution below.

The distribution of X
x01234
P(X = x).10.25.35.20.10
  1. Show that this is a legitimate probability distribution.
  2. Find the mean μ.
  3. Find the variance and standard deviation, by the definition and by the shortcut.
  4. What is P(X ≥ 2)?
  5. Would μ = np have given the mean more quickly?
Worked solution
  1. Two conditions to check Every P(X = x) lies between 0 and 1 ✓ .10 + .25 + .35 + .20 + .10 = 1.00 ✓
  2. Mean: μ = Σ x · p 0(.10) + 1(.25) + 2(.35) + 3(.20) + 4(.10) = 0 + .25 + .70 + .60 + .40 = 1.95 bikes The mean of a random variable need not be a value the variable can take. No hour ever sees 1.95 bikes returned.
  3. Variance by the definition: σ² = Σ (x − μ)² · p (0−1.95)²(.10) = 3.8025 × .10 = .380250 (1−1.95)²(.25) = 0.9025 × .25 = .225625 (2−1.95)²(.35) = 0.0025 × .35 = .000875 (3−1.95)²(.20) = 1.1025 × .20 = .220500 (4−1.95)²(.10) = 4.2025 × .10 = .420250 σ² = 1.2475
  4. Variance by the shortcut: σ² = E(X²) − μ² E(X²) = 0(.10) + 1(.25) + 4(.35) + 9(.20) + 16(.10) = 5.05 σ² = 5.05 − 1.95² = 5.05 − 3.8025 = 1.2475 ✓ σ = √1.2475 = 1.117 bikes The two routes must agree exactly, and the second is far less error-prone on a calculator, because it never subtracts the mean five separate times.
  5. A tail probability is a sum over the table P(X ≥ 2) = .35 + .20 + .10 = .65
  6. Why the binomial shortcut does not apply μ = np and σ = √(np(1 − p)) belong to a binomial count, which needs a fixed number of independent trials each carrying the same success probability. Nothing here supplies an n or a p: the table was measured, not derived from trials. So the general formulas are the only route. Read the other way, the binomial shortcut is what you get when you run these same sums on a distribution that does have that structure.

Valid distribution; μ = 1.95, σ² = 1.2475, σ = 1.117, P(X ≥ 2) = .65. The binomial shortcut needs a fixed n and a constant p, and this distribution has neither.

From §1.8 The Binomial Distribution · the same two sums, run on a binomial, give np and √(np(1 − p)).

16Fifteen visitors in one room

Exam-level

A museum times how long each of fifteen visitors spends in one small exhibition room, to the nearest minute. The times are already sorted.

Minutes in the room, fifteen visitors 3, 6, 7, 9, 10, 12, 12, 14, 15, 16, 18, 19, 21, 24, 41
  1. Give the five-number summary, using the textbook rule for the quartiles.
  2. Apply the 1.5 × IQR rule. Where are the fences, and which times fall outside them?
  3. Sketch the boxplot: say where the box edges, the median line, the two whisker ends and any separate points go.
  4. The same data in R returns Q1 = 9.5 and Q3 = 18.5. Does the flagged set change?
Worked solution
  1. Median first, because the quartiles are built from it n = 15 is odd, so the median is the 8th value. Median = 14 min
  2. Quartiles: median of each half, the median itself left out Lower half (7 values): 3, 6, 7, 9, 10, 12, 12 → Q1 = 4th = 9 Upper half (7 values): 15, 16, 18, 19, 21, 24, 41 → Q3 = 4th = 19 IQR = 19 − 9 = 10 min With an odd n the middle value belongs to neither half. Include it in both and you get different quartiles, which is one of the two or three most common ways to lose a mark here.
  3. Five-number summary Min = 3 · Q1 = 9 · Median = 14 · Q3 = 19 · Max = 41
  4. Fences, then flags Lower: Q1 − 1.5 × IQR = 9 − 15 = −6 Upper: Q3 + 1.5 × IQR = 19 + 15 = 34 Only 41 lies outside [−6, 34] A negative lower fence is fine and quite common. It says nothing in this batch could have been low enough to flag, since the times cannot go below zero.
  5. The boxplot Box from 9 to 19, with the median line at 14, sitting left of center inside the box. The left whisker runs to the minimum, 3. The right whisker stops at 24, the largest value still inside the upper fence, not at the maximum. The 41 is drawn as its own point beyond the whisker.
  6. The software answer R: Q1 = 9.5, Q3 = 18.5, IQR = 9 Fences: 9.5 − 13.5 = −4 and 18.5 + 13.5 = 32 The quartiles differ because R interpolates between neighboring values instead of taking the median of a half. The box would be drawn very slightly narrower, and 41 is still the only flagged time. Different convention, same conclusion here, which is the usual outcome and exactly why it is worth knowing that both exist.

Five-number summary 3, 9, 14, 19, 41. Fences at −6 and 34, so the 41 is the single flagged time. Box 9 to 19 with the median at 14, whiskers to 3 and 24, and 41 as a separate point. R's quartiles (9.5 and 18.5) move the fences to −4 and 32 and flag the same value.

Rounding: none needed. Every quartile here lands on a data value or halfway between two, so the arithmetic is exact.

From §1.3 Describing Data and §1.4 Visualizing Data · check it in the Descriptives calculator, prefilled, where the Quartiles switch turns the textbook answer into R's.

17Nine readings, one blood pressure

Exam-level

A clinic's automatic monitor comes with a calibration certificate: repeated readings on the same person vary with a standard deviation of 4 mmHg, and that figure is treated as known. A patient is measured nine times in one sitting and the readings average 138 mmHg.

Known σ, sample size, sample mean σ = 4 mmHg · n = 9 · x̄ = 138 mmHg
  1. Give the standard error and the 95% z-interval for this patient's true mean pressure.
  2. Rebuild it at 99% confidence. How much wider is it, and what bought the extra width?
  3. The clinic wants a 95% margin of error no larger than 1.5 mmHg. How many readings does that take?
  4. A colleague reads the 95% interval as "a 95% probability the true mean lies between those two numbers". Say what is wrong with that sentence.
Worked solution
  1. Standard error, then the margin σ is known, so this is a z-interval and nothing is estimated in the denominator. SE = σ/√n = 4/√9 = 4/3 = 1.3333 mmHg m = z* × SE = 1.960 × 1.3333 = 2.6133 mmHg
  2. The 95% interval 138 ± 2.6133 = [135.39, 140.61] mmHg
  3. The same data at 99% m = 2.576 × 1.3333 = 3.4347 mmHg 138 ± 3.4347 = [134.57, 141.43] mmHg The width goes from 5.23 to 6.87 mmHg, about 31% wider, which is exactly the ratio 2.576/1.960. Nothing about the data changed. The extra width is the price of a procedure that misses one time in a hundred instead of one time in twenty.
  4. How many readings for m ≤ 1.5 Rearrange m = z*σ/√n for n. n = (z*σ/m)² = (1.960 × 4 / 1.5)² = 5.2267² = 27.32 Round up: n = 28 readings The rounding direction is worth a mark on its own. With 27 readings the margin comes out at 1.5088, which is over the limit; with 28 it is 1.4816. Rounding down breaks the promise the clinic just made.
  5. What the interval does not say The true mean is a fixed number, not a random one. This particular interval either contains it or it does not, and no probability attaches to that fact once the data are in. The 95% belongs to the procedure: build intervals this way forever and 95% of them capture the truth. Read the number as a statement about the method, and the sentence becomes true.

SE = 1.3333 mmHg. The 95% interval is 138 ± 2.61 = [135.39, 140.61] and the 99% interval is 138 ± 3.43 = [134.57, 141.43], about 31% wider. A margin of 1.5 mmHg needs n = 28 readings. The "95% probability" reading is wrong because confidence describes the long-run behavior of the procedure, not any one interval.

Rounding: z* values taken as the printed 1.960 and 2.576. Intervals given to 2 dp; the sample size is always rounded up.

From §1.11 Confidence Intervals · watch it assemble in the Interval Builder, prefilled, and read z* off the printed Table D.

18Bulbs that may be dimmer than promised

Exam-level

A supplier advertises a mean lifetime of 1,200 hours for its LED bulbs. Years of testing across the industry put the standard deviation at 100 hours, and it is treated as known. A consumer group suspects the bulbs do not last as long as claimed, buys 25 of them and burns them out. The 25 last 1,165 hours on average.

Claim, known σ, sample μ₀ = 1,200 h · σ = 100 h · n = 25 · x̄ = 1,165 h
  1. Write H₀ and H₁. The wording decides how many tails; say which words did it.
  2. Compute the standard error and the test statistic.
  3. Find the p-value and give the decision at α = .05 and at α = .01.
  4. A colleague argues the test should have been two-sided. Recompute the p-value that way, and say why the tail cannot be picked after the data arrive.
Worked solution
  1. Hypotheses, from the wording H₀: μ = 1,200    H₁: μ < 1,200 "Do not last as long as claimed" names a direction, so this is a left-tailed test. Had the group asked whether the advertised figure was simply wrong, H₁ would be μ ≠ 1,200.
  2. Standard error and statistic SE = σ/√n = 100/√25 = 100/5 = 20 h z = (x̄ − μ₀)/SE = (1,165 − 1,200)/20 = −35/20 = −1.75
  3. The p-value, left tail One tail, so read the area below −1.75 straight from Table A. p = P(Z < −1.75) = .0401
  4. Two decisions, two thresholds At α = .05: .0401 < .05, so reject H₀ At α = .01: .0401 > .01, so fail to reject H₀ The evidence did not change between those two lines. The same z = −1.75 sits past the .05 cutoff of −1.645 and short of the .01 cutoff of −2.326. This is why α belongs in the write-up: "significant" means nothing without the threshold it was judged against.
  5. The two-sided version p = 2 × P(Z < −1.75) = 2 × .0401 = .0801 Doubling it pushes the result past .05, so a two-sided test keeps H₀ at both thresholds. That is the whole reason the tail has to be fixed in advance. Choosing it once the sample mean is on the table means always taking the half that helps, which turns a nominal 5% false-alarm rate into a real 10% one.
  6. The sentence to write At α = .05, one-tailed: the 25 bulbs give evidence that mean lifetime falls short of the advertised 1,200 hours, z = −1.75, p = .040. The shortfall estimated here is 35 hours, which is worth reporting beside the p-value, since a 3% shortfall may or may not matter to a buyer.

H₀: μ = 1,200 against H₁: μ < 1,200, one-tailed because the group suspected a shortfall. SE = 20, z = −1.75, one-tailed p = .0401: reject at α = .05, fail to reject at α = .01. Two-sided the p-value is .0801, which rejects at neither.

Rounding: p-values to 4 dp from Table A; a printed table gives .0401 for z = −1.75 directly.

From §1.12 Hypothesis Testing Logic · load it into the explorer, prefilled, or look the area up on the printed Table A.

19Sixteen shots off one machine

Exam-level

A café is checking whether its espresso machine is pulling shots to the recipe, which calls for an extraction time of 25 seconds. Over one morning the barista times sixteen shots. They average 27.4 seconds with a sample standard deviation of 3.2 seconds. A histogram of the sixteen times leans mildly to the right and holds no unusual values.

Sample size, mean, sample SD, recipe value n = 16 · x̄ = 27.4 s · s = 3.2 s · μ₀ = 25 s
  1. Give the standard error of the mean and the 95% confidence interval for the true mean extraction time.
  2. Read the verdict of a two-sided test at α = .05 off that interval, then confirm it by computing t and bracketing its p-value on Table D.
  3. Rebuild the interval at 99% confidence. Does the conclusion change?
  4. The times are mildly skewed. Is a t procedure defensible on sixteen observations, and which band of the rule of thumb decides it?
Worked solution
  1. Standard error, then the multiplier σ is unknown and s has been estimated from the same sixteen shots, so this is a t procedure with df = n − 1 = 15. SE = s/√n = 3.2/√16 = 3.2/4 = 0.800 s Table D, row 15, 95% column: t* = 2.131
  2. The 95% interval m = t* × SE = 2.131 × 0.800 = 1.705 s 27.4 ± 1.705 = [25.70, 29.11] s
  3. The verdict, read off the interval 25 lies below 25.70, so it is outside the interval. A 95% interval holds exactly the values a two-sided test at α = .05 would not reject, so the test rejects H₀: μ = 25. t = (x̄ − μ₀)/SE = (27.4 − 25)/0.800 = 3.00 On row 15 of Table D, 3.00 falls between the .005 column (2.947) and the .0025 column (3.286), so the upper-tail area is between .0025 and .005 and the two-tailed p is between .005 and .01. Exactly, p = .0090. The interval and the test agree, as they must.
  4. The same data at 99% t*(15, 99%) = 2.947 → m = 2.947 × 0.800 = 2.357 s 27.4 ± 2.357 = [25.04, 29.76] s Still above 25, though only just. The two-tailed p of .0090 is under .01, which says the same thing: this result also clears the stricter threshold. Had p landed at, say, .03, the 99% interval would have swallowed 25 while the 95% one did not.
  5. Is t allowed here Sixteen observations put this in the middle band of the rule of thumb, 15 to 40, where mild skewness is acceptable and outliers are not. The description gives mild skew and no unusual values, so the procedure is defensible. Under 15 observations the same data would have needed a shape much closer to normal, and a single wild shot would have been a problem at any sample size, because it moves the mean and inflates s at once.
  6. The sentence to write Sixteen shots averaged 27.4 s (SD = 3.2), significantly longer than the 25 s the recipe calls for, t(15) = 3.00, p = .009, 95% CI [25.70, 29.11]. The interval is the useful half of that sentence: the machine is running long by somewhere between 0.7 and 4.1 seconds, and whether that matters is a question about coffee rather than about statistics.

SE = 0.800 s. The 95% interval is 27.4 ± 1.71 = [25.70, 29.11], which excludes 25, so a two-sided test rejects at α = .05; t(15) = 3.00 with p between .005 and .01 on Table D (.0090 exactly). At 99% the interval is [25.04, 29.76] and still excludes 25. With n = 16, mild skew and no outliers, the middle band of the rule of thumb allows the t procedure.

Rounding: t* values as Table D prints them (2.131 and 2.947); interval limits to 2 dp.

From §1.13 One-Sample & Paired t-Tests · mark t = 3.00 on the t-distribution viz, read t* off the printed Table D, or paste a column into the descriptives calculator.

20Two warm-ups, two degrees of freedom

Stretch

A sports scientist compares two warm-up routines by measuring vertical jump height. Twelve athletes do the dynamic routine and fifteen do the static one; nobody does both. The dynamic group averages 46.8 cm with a sample standard deviation of 5.4 cm, the static group 41.9 cm with a standard deviation of 4.1 cm. Working by hand, then checking the result in software, produces two different degrees of freedom.

Two independent groups, summary statistics only n₁ = 12 · x̄₁ = 46.8 · s₁ = 5.4   |   n₂ = 15 · x̄₂ = 41.9 · s₂ = 4.1
  1. Compute the standard error of the difference between the two means, and the t statistic.
  2. Using the conservative degrees of freedom, give the critical value at α = .05 two-tailed, the decision, and the bracket Table D puts the p-value in.
  3. Software reports Welch degrees of freedom of 20.12 for the same data. Give the p-value there. Do the two answers agree, and would they always?
  4. Give the 95% confidence interval for the difference, using the conservative rule.
  5. Which answer should you write in an exam, and what makes the conservative rule safe to accept?
Worked solution
  1. Standard error of the difference Each group contributes its own variance over its own n. The two contributions add before the square root; the two standard errors are never added. s₁²/n₁ = 29.16/12 = 2.4300    s₂²/n₂ = 16.81/15 = 1.1207 SE = √(2.4300 + 1.1207) = √3.5507 = 1.884 cm t = (46.8 − 41.9)/1.884 = 4.90/1.884 = 2.600
  2. The conservative degrees of freedom df = min(n₁ − 1, n₂ − 1) = min(11, 14) = 11 Table D, row 11, 95% column: t* = 2.201 2.600 is beyond 2.201, so H₀ of equal means is rejected at α = .05. On the same row, 2.600 sits between the .02 column (2.328) and the .01 column (2.718). Those headings are one-tail areas, so the one-tail figure lands between .01 and .02, and doubling puts the two-tailed p between .02 and .04. Exactly, p = .0247.
  3. The same statistic on the software's curve Nothing about the data changed: the standard error is still 1.884 and t is still 2.600. Only the curve those numbers are read against moves, from 11 degrees of freedom to 20.12. df = 20.12 → p = .0171 Both reject at α = .05, so the two answers agree here. They would not always. A statistic sitting between the two critical values, anywhere from 2.085 to 2.201 on this data, clears the software's threshold and misses the hand rule's, which is exactly the direction the conservative rule is designed to err in.
  4. The interval for the difference 4.90 ± 2.201 × 1.884 = 4.90 ± 4.147 95% CI = [0.75, 9.05] cm Zero is outside it, which is the rejection restated. The width is the more honest finding: twenty-seven athletes locate the advantage of the dynamic routine somewhere between three quarters of a centimeter and nine, and only the top of that range would interest a coach. At the Welch df the interval tightens slightly, to [0.97, 8.83].
  5. Which one to write down In an exam, the conservative one. It needs nothing beyond a square root, and every error it makes runs the same way: holding the fewest degrees of freedom of the three, it draws the fattest tails, so its critical value is the highest and its p the biggest. A result that clears that bar would have cleared either of the others too, which is why a marker can take it on trust. Away from the exam hall, quote whichever df your software printed and name which one it was.

SE = 1.884 cm and t = 2.600. At the conservative df = 11, t* = 2.201, so H₀ is rejected at α = .05, with Table D bracketing the two-tailed p between .02 and .04 (.0247 exactly). At the Welch df of 20.12 the p is .0171; both reject, though they need not always. The conservative 95% CI for the difference is [0.75, 9.05] cm. Write the conservative answer in an exam, because it errs only toward caution.

Rounding: SE and t to three decimal places as a calculator gives them; t* as Table D prints it; interval limits to 2 dp. Pooling instead, which these unequal spreads do not obviously justify, would give sp = 4.716, t = 2.683 on 25 df and p = .0128.

From §1.14 Independent Samples t-Test · compare the three rules live in the Tug-of-War with the conservative df selected, or look t* up on the printed Table D.

21Two hundred riders, three verdicts

Exam-level

A bike-share operator has replaced the docking stations across a city and asks 200 riders whether the new ones are better, about the same, or worse than the old. The riders split into 120 annual members and 80 casual users. The operator wants to know whether the two kinds of rider are answering differently.

A 2 × 3 table of counts Members: better 70 · same 32 · worse 18   |   Casual: better 30 · same 28 · worse 22
  1. Give the row totals, the column totals and the grand total, then the six expected counts. Do the textbook conditions hold?
  2. Compute the six components and add them to get χ².
  3. State the degrees of freedom, bracket the p-value off Table F, and give the decision at α = .05.
  4. Which cell contributes most, and what does that cell say about the two kinds of rider?
  5. Give each row's conditional distribution as percentages, and write the finding in one sentence.
Worked solution
  1. Margins first, then the expected counts Every expected count is its own row total times its own column total, divided by the grand total. Doing the margins first means each one is a single multiplication and a single division. Rows: 70 + 32 + 18 = 120   30 + 28 + 22 = 80    Columns: 100, 60, 40    N = 200 Members: 120 × 100/200 = 60   120 × 60/200 = 36   120 × 40/200 = 24 Casual: 80 × 100/200 = 40   80 × 60/200 = 24   80 × 40/200 = 16 Check the margins of the expected table before going on: 60 + 36 + 24 = 120 and 40 + 24 + 16 = 80, so the expected counts reproduce the observed totals, which they always must. The smallest expected count is 16, so every one clears 1, none is below 5, and the conditions hold comfortably.
  2. One component per cell Subtract, square, divide by the expected count. Keep four decimal places until the final addition. Members: (70 − 60)²/60 = 1.6667   (32 − 36)²/36 = 0.4444   (18 − 24)²/24 = 1.5000 Casual: (30 − 40)²/40 = 2.5000   (28 − 24)²/24 = 0.6667   (22 − 16)²/16 = 2.2500 χ² = 1.6667 + 0.4444 + 1.5000 + 2.5000 + 0.6667 + 2.2500 = 9.0278
  3. Degrees of freedom, then Table F df = (2 − 1) × (3 − 1) = 2 On the df 2 row, 9.0278 sits between the .02 column (7.824) and the .01 column (9.210), so the p-value is between .01 and .02. Since that whole range is below .05, reject the hypothesis that rider type and verdict are independent. Exactly, p = .0110.
  4. Where the 9.03 came from The largest component is 2.50, casual riders who said better. Independence expects 40 of them and only 30 said so. The second largest, 2.25, is its mirror image: 22 casual riders called the stations worse where 16 were expected. Members run the other way in both cells. So the finding is not that riders are split on the docks in general, it is that casual users are markedly less enthusiastic than members, and both ends of their answer distribution show it. The smallest component, 0.44, says the middle option is answered almost identically by the two groups.
  5. The two conditional distributions A significant χ² is a claim that these two rows differ. Divide each row by its own total and the claim becomes readable. Members: 70/120 = 58.3%   32/120 = 26.7%   18/120 = 15.0% Casual: 30/80 = 37.5%   28/80 = 35.0%   22/80 = 27.5% Everybody: 100/200 = 50%   60/200 = 30%   40/200 = 20% Members split 58 / 27 / 15 and casual users 38 / 35 / 28, against the 50 / 30 / 20 of the survey as a whole. Independence would have both rows sitting on that last line. The sentence to write: members are the ones happy with the new docks, and approval falls off steeply among casual users, a quarter of whom call the stations worse.

Expected counts 60, 36, 24 for members and 40, 24, 16 for casual riders; all conditions met. Components 1.6667, 0.4444, 1.5000, 2.5000, 0.6667 and 2.2500 give χ² = 9.03 on 2 df. Table F brackets p between .01 and .02 (p = .0110 exactly), so rider type and verdict are associated at α = .05. The biggest contributor is casual riders who chose "better", 30 against an expected 40. Row percentages: members 58.3 / 26.7 / 15.0, casual 37.5 / 35.0 / 27.5.

Rounding: components to 4 dp so the sum is right in the second decimal; χ² to 2 dp and percentages to 1 dp in the report. Reporting it: χ²(2, N = 200) = 9.03, p = .011. Problem 19 takes a different 2 × 3 table on to Cramér's V.

From §1.16 Chi-Square Tests · type the table into the Build a Table explorer to see the same components, the same row percentages and the tint, or look the critical values up on the printed Table F.

22Two reminders, one gap in attendance

Exam-level

A clinic sends appointment reminders two ways and records who turns up. Of 250 patients reminded by text message, 205 attended. Of 200 reminded by email, 148 attended. The practice manager wants to know whether the text reminder really does better, and by how much.

Two independent groups, counts of successes n₁ = 250 · x₁ = 205 (text)   |   n₂ = 200 · x₂ = 148 (email)
  1. Check the conditions, then give the two sample proportions and their difference.
  2. Compute the pooled proportion and the standard error the test uses, then the z statistic and its two-sided p-value.
  3. Give the 95% confidence interval for the difference.
  4. The interval's standard error is not the test's. Why not, and which one belongs where?
  5. Write the result as a sentence, and say what χ² on the same 2 × 2 table would give.
Worked solution
  1. Conditions, then the proportions Each group needs at least 10 successes and 10 failures. Here it is 205 and 45 in one group, 148 and 52 in the other, so there is nothing to worry about on that front. Both samples are also tiny next to the population of patients a clinic sees. p̂₁ = 205/250 = 0.820    p̂₂ = 148/200 = 0.740    p̂₁ − p̂₂ = 0.080
  2. The test, which believes H₀ H₀ says the two population proportions are equal. If that is true there is only one proportion to estimate, and every one of the 450 patients helps estimate it. That single pooled value goes into both halves of the standard error. p̂ = (205 + 148)/(250 + 200) = 353/450 = 0.7844 SE = √(0.7844 × 0.2156 × (1/250 + 1/200)) = √(0.16909 × 0.00900) = √0.0015218 = 0.03901 z = 0.080 / 0.03901 = 2.051    two-sided p = 2 × P(Z > 2.05) = .0403 Table A gives the area left of 2.05 as .9798, so the upper tail is .0202 and doubling it gives .0404. Reject H₀ at α = .05, though not at .01.
  3. The interval, which believes nothing An interval has no hypothesized value to lean on, so each group estimates its own spread from its own data and the two variances add. SE = √(0.82 × 0.18/250 + 0.74 × 0.26/200) = √(0.0005904 + 0.0009620) = √0.0015524 = 0.03940 0.080 ± 1.96 × 0.03940 = 0.080 ± 0.077 95% CI = [0.003, 0.157]
  4. Why the two standard errors differ A proportion's spread is decided by the proportion itself, so choosing a value for p also chooses a standard error. The test assumes H₀ is true and H₀ hands it one shared proportion, so it would be inconsistent to test that hypothesis using two separate estimates. The interval assumes nothing at all and has only the two observed proportions to work with. The two standard errors come out at 0.03901 and 0.03940 here, close enough that the conclusion is safe either way, but the two formulas answer different questions and swapping them costs marks.
  5. The sentence, and the same test in its other costume Text reminders were followed by attendance more often than email reminders, 82.0% against 74.0%, z = 2.05, p = .040, 95% CI of the difference [0.3%, 15.7%]. Laid out as a 2 × 2 table of attended against missed, the same 450 patients give χ²(1, N = 450) = 4.21, and 4.2055 is 2.0507 squared. The p-value is .040 either way. The z version is the one to report, because the interval it comes with is the part the practice manager can act on: the advantage could be as small as a third of a percentage point.

p̂₁ = .820, p̂₂ = .740, difference .080. Pooled p̂ = .7844, test SE = .03901, z = 2.05, two-sided p = .040, so reject H₀ at α = .05. The 95% interval uses the unpooled SE = .03940 and runs [0.003, 0.157]. χ² on the same table is 4.21 on 1 df, which is z² exactly, with the same p.

Rounding: proportions to 3 dp, standard errors to 5 dp so the z survives to two decimals, interval limits to 3 dp. Reading z = 2.05 off Table A gives p = .0404 rather than .0403; both round to .040.

From §1.15 Inference for Proportions · set the counts in the One Proportion, or Two explorer, or check the areas on the printed Table A.

23Five wards, one waiting room

Exam-level

A hospital records, for five wards on the same weekday evening, how many nurses were on shift and the average time a patient waited to be seen. The manager wants a number for how strongly the two go together, computed by hand, because the report is due before anyone can get to a computer.

Five wards
Nurses on shift (x)246810
Mean wait, minutes (y)4534403031
  1. Compute both means and both sample standard deviations.
  2. Build the table of deviations, standardized values and their products, and compute r as the average of the products.
  3. Interpret the sign and the size, and state what r² adds.
  4. Which ward contributes nothing to r, and why?
Worked solution
  1. Means and standard deviations x̄ = 30 / 5 = 6.0 nurses    ȳ = 180 / 5 = 36.0 minutes Σ(x − x̄)² = 16 + 4 + 0 + 4 + 16 = 40    sx = √(40/4) = √10 = 3.162 Σ(y − ȳ)² = 81 + 4 + 16 + 36 + 25 = 162    sy = √(162/4) = √40.5 = 6.364
  2. The table the formula asks for
    xyx − x̄y − ȳ(x − x̄)/sx(y − ȳ)/syproduct
    245−4+9−1.265+1.414−1.789
    434−2−2−0.632−0.314+0.199
    6400+40.000+0.6290.000
    830+2−6+0.632−0.943−0.596
    1031+4−5+1.265−0.786−0.994
    Σ products = −3.180    r = −3.180 / (5 − 1) = −0.795 The shortcut route agrees, and is faster if you trust it: Σ(x − x̄)(y − ȳ) = −64, so r = −64 / √(40 × 162) = −64 / 96.00 = −0.795. Use whichever the question asks for; the standardized table is the one that shows your working.
  3. Interpretation The sign is negative, so more nurses goes with shorter waits. The size, |r| = .80, is a strong linear association by any of the usual benchmarks, though five wards is a very thin basis for it. r² = (−0.795)² = 0.632, so about 63% of the variation in waiting times across these five wards lines up with the variation in staffing, leaving 37% to everything else: how sick the arriving patients were, how the ward is laid out, and plain chance.
  4. The ward that contributes nothing The six-nurse ward. Its standardized x is exactly zero, because it sits on the mean staffing level, and zero times its standardized y of +0.629 is zero. It waited four minutes longer than average and the coefficient never hears about it. Influence in a correlation comes from distance from BOTH means, which is the same arithmetic that makes r so easy for a single far-out point to move.

x̄ = 6.0, ȳ = 36.0, sx = 3.162, sy = 6.364. The five standardized products are −1.789, +0.199, 0.000, −0.596 and −0.994, totalling −3.180, so r = −3.180/4 = −0.795. r² = 0.632. The six-nurse ward contributes nothing, because its standardized x is zero.

Rounding: standardized values and products to 3 dp, r to 3 dp. Carrying only 2 dp through the table gives −0.794, which still rounds to −0.79.

From §1.17 Correlation · step the same shape through the 📏 Correlation, Step by Step viz, or load these five wards straight into the explorer: ?data=2:45,4:34,6:40,8:30,10:31.

24The line from five summary numbers

Exam-level

A study of 60 first-year students records how many hours each slept the night before an exam and the mark they went on to get. The raw data is not in the report; the five numbers below are all the report prints. Sleep in the sample ranged from 4.5 to 9.0 hours.

Summary statistics, n = 60 x = hours slept: x̄ = 6.8, sx = 1.2   |   y = exam mark: ȳ = 63.5, sy = 11.4   |   r = .42
  1. Find the least-squares line for predicting mark from sleep.
  2. Predict the mark for a student who slept 8 hours, and interpret the slope in the study's own units.
  3. What does the intercept mean here, and how much should anyone rely on it?
  4. A colleague wants the line for predicting sleep from mark instead, and suggests inverting the slope you already have. Give the right answer and say why the shortcut fails.
Worked solution
  1. Slope, then intercept b₁ = r · sy / sx = 0.42 × 11.4 / 1.2 = 0.42 × 9.5 = 3.99 b₀ = ȳ − b₁ x̄ = 63.5 − 3.99 × 6.8 = 63.5 − 27.132 = 36.368 ŷ = 36.37 + 3.99x
  2. Prediction and slope in words ŷ = 36.368 + 3.99 × 8 = 36.368 + 31.92 = 68.29 A student sleeping 8 hours is predicted to score about 68 marks. The slope says that one extra hour of sleep goes with about 4 more marks, on average, across these 60 students. It is an association and not a promise: nobody was assigned a bedtime, so a student who sleeps well may differ in other ways that also help.
  3. The intercept b₀ = 36.4 is the mark the line predicts for a student who slept zero hours. Nobody in this sample slept less than 4.5, so that value sits well outside the data and the line has no evidence behind it there. Read it as the arithmetic that pins the line to (x̄, ȳ) rather than as a claim about sleepless students. The honest range for prediction is 4.5 to 9.0 hours, where the line runs from 54.3 to 72.3 marks.
  4. Swapping x and y b₁′ = r · sx / sy = 0.42 × 1.2 / 11.4 = 0.0442 hours per mark The inversion would have given 1/3.99 = 0.251, nearly six times too big. Correlation is symmetric and regression is not: the two slopes multiply to r² = 0.1764, not to 1, and they agree only when r = 1. Least squares minimizes vertical distances, and swapping the axes changes which direction "vertical" points, so it is genuinely a different line.

ŷ = 36.37 + 3.99x. At 8 hours the prediction is 68.3 marks, and each extra hour of sleep is worth about 4 marks. The intercept 36.4 is the predicted mark at zero hours, far outside the observed 4.5 to 9.0 range and not interpretable. Reversing the roles gives b₁′ = 0.0442 hours per mark, not 1/3.99 = 0.251, because b₁ · b₁′ = r² = .1764.

Rounding: the slope is exact at 3.99 because 11.4/1.2 = 9.5 exactly; carry the intercept to 3 dp (36.368) before rounding the printed line to 2 dp.

From §1.18 Simple Linear Regression · the same identity is in the fourth block of the formula sheet, and the Correlation & Regression Calculator fits a line to raw columns when you have them.

25Half the variation, and still the wrong shape

Exam-level

A grower tests eight doses of fertilizer, one pot per dose, and counts the tomatoes each pot produces. Software fits the least-squares line ŷ = 17.0 + 2.0x and reports R² = .500, which the grower takes as a decent result and starts writing up.

Eight pots, and the fitted line ŷ = 17.0 + 2.0x
Dose, grams (x)12345678
Tomatoes (y)1220263032323026
  1. Compute the eight predicted values and the eight residuals, and check their total.
  2. Sketch the residuals against dose. Name the pattern and say what it means about the line.
  3. R² is .500. Reconcile that with your answer to part 2.
  4. A second grower's residual plot is a wedge instead, narrow at low doses and wide at high ones. Name that pattern and say what it costs.
Worked solution
  1. Predictions and residuals
    x12345678
    ŷ = 17 + 2x1921232527293133
    y1220263032323026
    residual y − ŷ−7−1+3+5+5+3−1−7
    Σ residuals = (−7 − 1 + 3 + 5) + (5 + 3 − 1 − 7) = 0 The total is zero, and it always is for a least-squares line, so it confirms the arithmetic without saying anything about the fit.
  2. The pattern Plotted against dose, the residuals rise from −7 to +5 and fall back to −7: a clean arch, negative at both ends and positive through the middle. That is the signature of a curved relationship fitted with a straight line. Yield climbs with dose, flattens, then falls off as the dose becomes too much, and no straight line can bend to follow it. The line is too high at both ends and too low in the middle, by up to seven tomatoes.
  3. Why R² does not notice R² measures how much of the variation the line removes, not whether what is left has a shape. Here Σ(y − ȳ)² = 336 and the residuals leave 49 + 1 + 9 + 25 + 25 + 9 + 1 + 49 = 168, exactly half, so R² = 1 − 168/336 = .500 and r = .707. A rising line does capture the general upward drift over most of the range, which is enough for half the variation. The half it leaves behind is not noise; it is the curve. This is exactly why a residual plot is worth drawing even when the summary number looks acceptable, and it is Anscombe's second dataset in a different costume.
  4. The other grower's wedge A wedge is non-constant spread, which regression calls heteroscedasticity. The line's shape may be perfectly right; what fails is the assumption that the scatter around it is the same size everywhere. Predictions at the wide end are much less reliable than the narrow end, and the standard errors, confidence intervals and p-values that Stats 2 computes all rest on one shared spread, so they come out wrong for both ends at once. The usual repairs are a transformation of the outcome or a standard error that does not assume constant spread.

Predicted values 19, 21, 23, 25, 27, 29, 31, 33; residuals −7, −1, +3, +5, +5, +3, −1, −7, summing to zero. The residual plot is an arch, so the true relationship is curved and a straight line is the wrong shape. R² = .500 anyway, because R² measures variation removed rather than pattern left behind. The wedge is heteroscedasticity, which invalidates the standard errors rather than the shape of the line.

Rounding: none needed. Every predicted value and every residual is a whole number here, and R² is exactly .500 because Σ residual² = 168 is exactly half of Σ(y − ȳ)² = 336.

From §1.18 Simple Linear Regression · see the arch for yourself by loading these eight pots with the residual plot open: ?data=…&resid=1, then press "Snap to best fit". The full treatment is §2.4 Regression Diagnostics.

📝 Exam-style · A midterm on §1.1–§1.12

Eight problems written the way a paper exam asks them: one short scenario, several parts that build on each other, numbers a basic calculator can hold, and every area read off the printed tables rather than out of software. They span the first twelve sections of Stats 1, which is the usual reach of a week 4 or week 5 midterm. The set above teaches one idea per problem; this one chains them.

26Four words that must not be swapped

Warm-up

A university with 18,400 undergraduates wants to know how much sleep its students get on a weeknight. The registry draws a simple random sample of 250 names from the enrollment list and emails them a one-question form. 175 forms come back usable.

What the registry has
  • Undergraduates enrolled: 18,400
  • Names sampled: 250 · usable replies: 175
  • Reported sleep: mean 6.8 hours, standard deviation 1.3 hours
  • Replies reporting under 7 hours: 63
  1. Name the population, the sample, the parameter and the statistic for the sleep question.
  2. Write the right symbol for each of four quantities: the mean weeknight sleep of all 18,400; the 6.8; the true share of students sleeping under 7 hours; the share observed here.
  3. Compute the observed share.
  4. 75 of the 250 never replied. What does that put at risk, and would sampling 2,000 names instead of 250 repair it?
  5. Is the sample small enough, relative to the university, for the usual standard-error formulas to apply?
Worked solution
  1. The four words Population: all 18,400 enrolled undergraduates. Sample: the 175 who returned a usable form. Parameter: the mean weeknight sleep of the whole population, one fixed number nobody has measured. Statistic: the 6.8 hours, computed from the sample, and a different number in every sample the registry could have drawn.
  2. Notation μ = mean sleep of all 18,400 (parameter, unknown) x̄ = 6.8 hours (statistic, computed) p = true share sleeping under 7 hours (parameter, unknown) p̂ = observed share (statistic, computed) The population gets a Greek letter or a plain p; the sample gets a bar or a hat. Writing μ where you mean x̄ is the single most expensive slip on an early exam, because every sentence after it is then about the wrong quantity.
  3. The observed share p̂ = 63 / 175 = 0.36 36% of the students who replied sleep under 7 hours. Note the denominator: 175, the people who answered, not the 250 who were asked.
  4. The 75 who did not reply The selection was random, but the participation was not. The 75 silent students chose themselves out, and whatever kept them from answering may well be related to sleep, so the 175 are no longer a random slice of the university. Raising the sample to 2,000 buys a more precise estimate of the same biased quantity: a bigger sample shrinks the standard error and leaves the bias exactly where it was. The repair is chasing the non-respondents, not adding more names.
  5. Sample size against population size 18,400 / 250 = 73.6, so the sample is about 1.4% of the university The usual caution is that the population should be at least ten or twenty times the sample. At 73.6 times, this one clears it comfortably, which is why the standard error depends on n alone and the size of the university never enters the formula.

Population: all 18,400 undergraduates. Sample: the 175 usable replies. Parameter: μ, their true mean sleep. Statistic: x̄ = 6.8 hours. The observed share is p̂ = 63/175 = 0.36. The 30% non-response is a bias that a larger sample cannot fix, and at 1.4% of the population the sampling fraction is small enough to ignore.

From §1.1 What Is Statistics? and §1.6 Producing Data & Sampling Design.

27Thirteen calls to the help desk

Exam-level

A university IT help desk logs how long each caller waited before a person picked up, to the nearest minute. Here are thirteen calls from one morning, in the order they arrived.

Waiting time in minutes, thirteen calls 14, 6, 22, 9, 31, 11, 7, 18, 12, 8, 26, 10, 47
  1. Sort the values and give the five-number summary, using the textbook rule for the quartiles.
  2. Apply the 1.5 × IQR rule. Where do the fences fall, and is the 47-minute call an outlier?
  3. The same thirteen numbers in R return Q1 = 9 and Q3 = 22. Redo part 2 with those and say what changes.
  4. The desk manager wants one number for a typical wait. Mean or median, and why?
Worked solution
  1. Sort first; nothing else works until you have 6, 7, 8, 9, 10, 11, 12, 14, 18, 22, 26, 31, 47 n = 13 is odd, so the median is the 7th value. Median = 12 min
  2. Quartiles: median of each half, the middle value in neither Lower half (6 values): 6, 7, 8, 9, 10, 11 → Q1 = (8 + 9)/2 = 8.5 Upper half (6 values): 14, 18, 22, 26, 31, 47 → Q3 = (22 + 26)/2 = 24 IQR = 24 − 8.5 = 15.5 min Five-number summary: 6 · 8.5 · 12 · 24 · 47
  3. Fences under the textbook quartiles 1.5 × IQR = 1.5 × 15.5 = 23.25 Lower: 8.5 − 23.25 = −14.75 Upper: 24 + 23.25 = 47.25 47 sits inside the upper fence by a quarter of a minute, so this rule flags nothing at all. The longest wait of the morning is, by this convention, an ordinary long wait.
  4. The same rule on R's quartiles IQR = 22 − 9 = 13, so 1.5 × IQR = 19.5 Fences: 9 − 19.5 = −10.5 and 22 + 19.5 = 41.5 Now 47 is outside, and the boxplot draws it as a separate point. Two defensible conventions, one dataset, opposite verdicts on the one call the manager cares about. It happens when a value sits near a fence, which is exactly where the argument always is. SPSS, incidentally, returns 8.5 and 24 here and agrees with the textbook rule, so the disagreement is not a simple hand-versus-software split.
  5. One number for a typical wait Mean = 221 / 13 = 17.0 min · Median = 12 min Five minutes apart on thirteen calls. The mean is carrying the 47 and the 31, and no caller that morning waited 17 minutes; the median is a wait a caller actually had. Report the median for a typical wait, and the maximum alongside it, since a 47-minute hold is the thing a manager needs to hear about and a median is designed not to mention it.

Five-number summary 6, 8.5, 12, 24, 47. Textbook fences −14.75 and 47.25 flag nothing; R's quartiles put the fences at −10.5 and 41.5 and flag the 47. Mean 17.0 against a median of 12: report the median, with the longest wait beside it.

Rounding: none. Every quartile lands halfway between two whole values, and 221/13 is exactly 17.

From §1.3 Describing Data and §1.4 Visualizing Data · load it into the Descriptives calculator, prefilled and flip the Quartiles switch to watch the flag appear.

28Bags that say 500 grams

Exam-level

A machine fills bags of coffee labeled 500 g. The weight it actually delivers is normally distributed with a mean of 502 g and a standard deviation of 4 g. One shift produces 240 bags. Work every part from the printed Table A, and give the answer the table gives.

The fill distribution X ~ N(μ = 502 g, σ = 4 g) · 240 bags on the shift · label weight 500 g
  1. What proportion of bags weigh less than the labeled 500 g?
  2. What proportion weigh more than 508 g?
  3. What proportion fall between 499 g and 506 g?
  4. About how many bags on this shift are under the label?
  5. Below what weight do the lightest 1% of bags sit?
Worked solution
  1. Under the label z = (500 − 502) / 4 = −0.50 Table A at z = −0.50 gives .3085 Table A always reports the area to the LEFT, which is what this question wants, so the entry is the answer with no arithmetic after it. About 30.85% of bags are under the label.
  2. Above 508 g z = (508 − 502) / 4 = 1.50 Table A at 1.50 gives .9332, the area BELOW P(X > 508) = 1 − .9332 = .0668
  3. Between 499 g and 506 g z₁ = (499 − 502)/4 = −0.75 → .2266 z₂ = (506 − 502)/4 = 1.00 → .8413 P(499 < X < 506) = .8413 − .2266 = .6147 A "between" question is two lookups and one subtraction, larger area first. Subtracting the z-scores instead of the areas is the standard way to lose this mark.
  4. How many of the 240 240 × .3085 = 74.04, so about 74 bags A proportion becomes a count by multiplying, and the answer is rounded to a whole bag because bags are whole.
  5. The lightest 1%: read the table backwards Hunt inside the body of Table A for the area nearest .0100. It carries .0099 at z = −2.33 and .0102 at z = −2.32, so −2.33 is the nearer entry. x = μ + zσ = 502 + (−2.33)(4) = 502 − 9.32 = 492.68 g The lightest 1% of bags weigh under about 492.7 g, roughly 7 g short of the label. Software solving for the exact quantile returns z = −2.3263 and x = 492.69 g; on paper the table's entry is the answer, and the gap between the two is a hundredth of a gram.

.3085 under the label, .0668 above 508 g, .6147 between 499 g and 506 g, about 74 of the 240 bags under the label, and the lightest 1% below 492.68 g.

Rounding: Table A carries four decimal places, so the three proportions are exact as read. The 1% cutoff inherits the table's two-decimal z and lands 0.01 g from the exact value.

From §1.5 Z-Scores & The Normal Distribution · every area here is on the printed Table A, and the look-it-up drill below it sets the same three moves at random.

29What de Méré got wrong

Exam-level

Two fair six-sided dice are rolled and both faces are recorded, so there are 36 equally likely outcomes. The second half of this problem is the pair of bets a seventeenth-century gambler took to Pascal, having noticed that one of them made money and the other did not.

Two events on one roll of two dice A = the two faces sum to 7 · B = at least one face shows a 6
  1. Find P(A), P(B), P(A and B), and P(A or B).
  2. De Méré's first bet was at least one 6 in four rolls of a single die, which he priced at 4 × 1/6 = 2/3. Say why that reasoning cannot be right, then give the correct probability.
  3. His second bet was at least one double 6 in 24 rolls of two dice. Compute it.
  4. How many rolls of two dice would the second bet have needed to be worth taking?
Worked solution
  1. Count outcomes, then apply the general addition rule A: (1,6) (2,5) (3,4) (4,3) (5,2) (6,1) → P(A) = 6/36 B: six rolls with a 6 first, six with a 6 second, the double 6 counted twice → 11/36 A and B: (1,6) and (6,1) → 2/36 P(A or B) = 6/36 + 11/36 − 2/36 = 15/36 = 5/12 = .4167 The overlap has to come back out once, because the two outcomes in it were counted inside both totals. Notice that B itself needed the same correction while it was being counted.
  2. Why 4 × 1/6 is not a probability Multiplying a probability by the number of trials is not one of the rules. Follow it to six rolls and it returns 6 × 1/6 = 1, certainty, though six rolls plainly can miss; follow it to seven and it returns more than 1. The reasoning is the addition rule applied to events that are not disjoint, since two of the four rolls can both show a 6. P(no 6 in four rolls) = (5/6)⁴ = 625/1296 = .4823 P(at least one 6) = 1 − 625/1296 = 671/1296 = .5177 "At least one" is the signal to work with the complement: one multiplication and one subtraction, instead of adding up four overlapping cases.
  3. The second bet P(no double 6 on one roll of two dice) = 35/36 P(no double 6 in 24 rolls) = (35/36)²⁴ = .5086 P(at least one double 6) = 1 − .5086 = .4914 Just under a half, where the first bet was just over. De Méré lost slowly on one and won slowly on the other, and the two probabilities differ by less than three points, which is why it took him a long run of evenings to notice.
  4. How many rolls he needed Want (35/36)ⁿ < .5, so n > ln(.5) / ln(35/36) = 24.6 n = 25: 1 − (35/36)²⁵ = .5055 One more roll turns the bet from a slow loss into a slow win. His mistake was a proportional one: four rolls of one die out of six faces felt like it should scale to 24 rolls of two dice out of 36 pairs, and that scaling is not what the complement rule does.

P(A) = 6/36, P(B) = 11/36, P(A and B) = 2/36, P(A or B) = 15/36 = .4167. The first bet pays .5177, not 2/3. The second pays .4914, and would have needed 25 rolls to clear a half.

Rounding: probabilities to 4 dp. The two exact fractions are 671/1296 and 1 − (35/36)²⁴.

From §1.7 Probability Basics · all 36 outcomes are laid out in the dice lab.

30Two batteries out of the drawer

Exam-level

A drawer holds 12 identical-looking AA batteries. Four of them are flat and eight still work. Two are taken out at random, one after the other, and the first is not put back.

The drawer 12 batteries · 4 flat, 8 good · two drawn without replacement
  1. Draw the probability tree and give P(both flat), P(neither flat) and P(at least one flat).
  2. What is P(the second is flat, given the first was flat)? What is P(the second is flat) if you know nothing about the first?
  3. Are the two draws independent? Say how you can tell before doing any arithmetic.
  4. Redo part 1 with the first battery put back. Which answers move, and which way?
Worked solution
  1. The tree, with the second layer conditioned on the first Flat then flat: (4/12)(3/11) = 12/132 = 1/11 = .0909 Flat then good: (4/12)(8/11) = 32/132 Good then flat: (8/12)(4/11) = 32/132 Good then good: (8/12)(7/11) = 56/132 = 14/33 = .4242 The four leaves come to 12 + 32 + 32 + 56 = 132 out of 132, which is the check worth doing before going any further. P(at least one flat) = 1 − 14/33 = 19/33 = .5758
  2. Two probabilities that look alike and are not P(2nd flat | 1st flat) = 3/11 = .2727 P(2nd flat) = 12/132 + 32/132 = 44/132 = 1/3 = .3333 The unconditional answer is 1/3, exactly what the first draw was. Before you look at the first battery, the second one is just as likely to be flat as the first; once you have seen the first come out flat, the drawer holds one fewer flat battery and the odds shorten.
  3. Independence, by inspection Not independent. Taking a battery out changes what is left, so the second draw runs on a different drawer depending on how the first went. You can say that without arithmetic: sampling without replacement from a finite pile is dependent by construction. The numbers confirm it, since .2727 is not .3333.
  4. With replacement P(both flat) = (1/3)² = 1/9 = .1111 P(neither flat) = (2/3)² = 4/9 = .4444 P(at least one flat) = 1 − 4/9 = 5/9 = .5556 Both matching outcomes get more likely and the mixed outcome gets less likely, because replacing removes the pull back toward balance: without replacement, drawing a flat one leaves the drawer richer in good ones. Every second-layer branch becomes 4/12 or 8/12 again, which is what independence means here.

Without replacement: P(both flat) = 1/11 = .0909, P(neither) = 14/33 = .4242, P(at least one flat) = 19/33 = .5758. P(2nd flat | 1st flat) = 3/11 against an unconditional 1/3, so the draws are dependent. With replacement the three become 1/9, 4/9 and 5/9.

Rounding: probabilities to 4 dp; every value here is an exact fraction.

From §1.7 Probability Basics.

31Eight questions, four options, no revision

Exam-level

A short class test has 8 multiple-choice questions with 4 options each and exactly one right answer per question. A student who has done no revision at all answers every question by guessing. Let X be the number correct. The pass mark is 5.

The guessing student 8 questions · 4 options each · X = number correct · pass mark 5 of 8
  1. Check the binomial conditions and name the distribution of X.
  2. Work out C(8, 3) by hand and use it for P(X = 3).
  3. Compute P(X ≤ 1).
  4. Give the mean and standard deviation of X.
  5. What is the probability of passing by guessing?
Worked solution
  1. The four conditions A fixed number of trials, n = 8. Two outcomes per trial, right or wrong. The guesses are independent, since one answer tells the student nothing about the next. And the success probability is the same each time, p = 1/4, because every question has four options. All four hold. X ~ B(n = 8, p = 0.25)
  2. C(8, 3), then the probability C(8,3) = 8! / (3! 5!) = (8 × 7 × 6) / (3 × 2 × 1) = 336 / 6 = 56 P(X = 3) = 56 × (0.25)³ × (0.75)⁵ = 56 × 0.015625 × 0.2373047 = .2076 Cancel before multiplying: 8 × 7 × 6 over 3 × 2 × 1 is three factors on top and three underneath, never the full 40,320. The three pieces are the number of ways to choose which questions come out right, the probability of getting those three right, and the probability of getting the other five wrong.
  3. P(X ≤ 1) is two terms P(X = 0) = (0.75)⁸ = .1001 P(X = 1) = 8 × (0.25) × (0.75)⁷ = 8 × 0.25 × 0.1334839 = .2670 P(X ≤ 1) = .1001 + .2670 = .3671 C(8,0) = 1 and C(8,1) = 8, so neither term needs the formula written out.
  4. Center and spread μ = np = 8 × 0.25 = 2 σ = √(np(1 − p)) = √(8 × 0.25 × 0.75) = √1.5 = 1.2247
  5. Passing by guessing P(X ≥ 5) = .0230713 + .0038452 + .0003662 + .0000153 = .0273 About 1 in 37. The pass mark of 5 sits (5 − 2)/1.2247 = 2.45 standard deviations above what guessing delivers on average, which is what a test designer is buying with four options rather than two: at two options the mean guess score would be 4 and half the room would pass on luck.

X ~ B(8, 0.25). C(8,3) = 56 and P(X = 3) = .2076. P(X ≤ 1) = .3671. μ = 2 correct, σ = 1.2247. The chance of passing by guessing is .0273, about 1 in 37.

Rounding: probabilities to 4 dp. The exact values are 13608/65536 for P(X = 3) and 6561/65536 for P(X = 0), and σ = √1.5 exactly.

From §1.8 The Binomial Distribution · see the whole distribution at n = 8, p = .25, and its shape beside the others in the distribution playground.

32One cup, then nine, then thirty-six

Exam-level

A vending machine is set to pour 250 ml of coffee. The amount it actually pours varies normally, with a mean of 250 ml and a standard deviation of 15 ml. The campus office treats anything under 244 ml as a short pour.

The machine Pour ~ N(μ = 250 ml, σ = 15 ml) · short pour means under 244 ml
  1. A student buys one cup. What is the probability it is short?
  2. An inspector buys 9 cups and averages them. What is the probability that average is under 244 ml?
  3. Repeat with 36 cups.
  4. Put the three probabilities side by side and say in one sentence why they fall the way they do.
  5. Which of the three answers needs the individual pours to be normally distributed, and which would survive a skewed machine?
Worked solution
  1. One cup: standardize the cup z = (244 − 250) / 15 = −0.40 P = .3446 The denominator is σ itself, because a single pour is what varies.
  2. Nine cups: standardize the mean SE = σ/√n = 15/√9 = 5 ml z = (244 − 250) / 5 = −1.20 P = .1151 Same 6 ml shortfall, divided by a smaller denominator, so it counts as a bigger departure. Using σ = 15 here instead of SE = 5 is the classic error, and it returns the answer to part 1.
  3. Thirty-six cups SE = 15/√36 = 2.5 ml z = −6 / 2.5 = −2.40 P = .0082
  4. Why .3446, .1151 and .0082 Averaging cancels: one stingy cup is unremarkable, nine cups averaging 6 ml low means nine pours leaned the same way at once, and thirty-six doing it is close to evidence that the machine itself has drifted rather than that the day was unlucky. Quadrupling n halves the standard error, which is why the step from 9 to 36 costs more than the step from 1 to 9 bought.
  5. Which answer leans on normality Part 1 is about a single observation, so it needs the pours themselves to be normal; there is no averaging to rescue it and no theorem that helps. Parts 2 and 3 are about a sample mean, where the central limit theorem does the work, but n = 9 is small enough to still lean on the population being roughly normal. At n = 36 the sampling distribution of the mean would be near normal even from a moderately skewed machine.

.3446 for one cup, .1151 for the mean of 9, .0082 for the mean of 36. The shortfall is the same 6 ml every time; what shrinks is the standard error, from 15 to 5 to 2.5. Only the first answer depends on the individual pours being normal.

Rounding: areas to 4 dp as Table A carries them. All three z-scores land on exact two-decimal table rows.

From §1.9 Sampling Distributions and §1.10 Central Limit Theorem · the three areas are on the printed Table A.

33The interval and the test, on one delivery

Stretch

A supplier certifies its resistors at a mean of 100 Ω, and the factory's long production records put the standard deviation at 6 Ω, treated as known. A buyer measures 36 resistors from one delivery and gets a mean of 102.2 Ω. The contract lets the buyer reject a delivery whose mean is off spec in either direction.

One delivery Claimed μ₀ = 100 Ω · known σ = 6 Ω · n = 36 · x̄ = 102.2 Ω · two-sided
  1. Build the 95% confidence interval for this delivery's mean.
  2. Run the two-sided z-test of H₀: μ = 100 at α = .05, taking the tail area from Table A.
  3. The interval and the test have to agree. Show that they do, and say why they must.
  4. Redo both at the 1% level. Do they still agree?
  5. Name the error the buyer could be making after each of the two decisions, and what it would cost.
Worked solution
  1. The 95% interval SE = σ/√n = 6/√36 = 1.0 Ω z* = 1.96, from the ∞ row of Table D margin = 1.96 × 1.0 = 1.96 Ω 102.2 ± 1.96 → [100.24, 104.16] 100 is not in that interval, which is already the answer to part 3 in disguise.
  2. The test z = (x̄ − μ₀)/SE = (102.2 − 100)/1.0 = 2.20 Table A at 2.20 gives .9861 below, so the upper tail is .0139 Two-sided p = 2 × .0139 = .0278 .0278 < .05, so reject H₀ Two-sided, because the contract lets the buyer complain about resistors that are too high or too low, and that decision was made by the contract rather than by looking at the 102.2.
  3. Why they cannot disagree The 95% interval is precisely the set of claimed means that a two-sided test at α = .05 would not reject. Both are asking whether μ₀ sits more than 1.96 standard errors from x̄: the interval measures that distance outward from the sample mean, the test measures the same distance inward from the claim. Here 100 is 2.20 standard errors away, which is past 1.96, so 100 falls outside the interval and the test rejects it. One piece of arithmetic, read two ways.
  4. The same delivery at the 1% level z* = 2.576 → 102.2 ± 2.576 → [99.62, 104.78] 100 IS inside this interval p = .0278 > .01, so fail to reject H₀ They agree again, in the other direction. Demanding 99% confidence widens the interval until it swallows the claim, and demanding a 1% significance level raises the bar until the same evidence no longer clears it. Nothing about the resistors changed between those two lines.
  5. Which error is live after each decision At α = .05 the buyer rejected, so the only error available is Type I: sending back a delivery that was on spec all along, and paying for a return and a delay over a batch that was fine. At α = .01 the buyer did not reject, so the live error is Type II: accepting a delivery whose mean really is about 2 Ω high, and building it into a product. Tightening α from .05 to .01 does not reduce the chance of being wrong; it trades one way of being wrong for the other, and which trade is right depends on which mistake costs more here.

SE = 1.0 Ω. The 95% interval is [100.24, 104.16] and excludes 100; z = 2.20 with a two-sided p of .0278 rejects at α = .05. At the 1% level the interval [99.62, 104.78] contains 100 and the test fails to reject, because .0278 > .01. Rejecting risks a Type I error, failing to reject risks a Type II.

Rounding: z* taken as 1.96 and 2.576, the values printed under Table D. The exact quantiles are 1.95996 and 2.57583, which shift each endpoint by under a hundredth of an ohm. The tail area .0139 is Table A's entry at z = 2.20.

From §1.11 Confidence Intervals and §1.12 Hypothesis Testing Logic · build the interval in the Interval Builder, prefilled, then run the same numbers through the test explorer, prefilled.

🔗 Stats 2 · Groups & relationships

Nineteen problems on comparing groups and reading a relationship: three or more groups, factorial designs, the assumptions that decide which test you are allowed to run, the two ways of describing a straight line, two ANOVA tables built by hand, the planned and corrected comparisons that follow one, and three rank-based tests worked against their printed critical-value tables. The group tests are Stats 2; correlation and regression are taught in Stats 1.

13Proofreading in three kinds of noise

Exam-level

Fifteen copy-editors are randomly assigned to proofread the same 2,000-word article in one of three sound conditions. The outcome is the number of planted errors they missed.

Errors missed, n = 5 per condition
  • Silence: 3, 5, 4, 2, 6
  • Instrumental music: 4, 7, 6, 5, 8
  • Café chatter: 9, 6, 8, 10, 7
  1. Build the full one-way ANOVA table.
  2. Decide at α = .05.
  3. Report η² and say what it means.
  4. Why not run three t-tests instead?
Worked solution
  1. Group means and the grand mean Silence: Σ = 20, M = 4 Music: Σ = 30, M = 6 Café: Σ = 40, M = 8 Grand: Σ = 90, N = 15, GM = 6
  2. Between-groups sum of squares SS_B = Σ n_j (M_j − GM)² = 5(4−6)² + 5(6−6)² + 5(8−6)² = 5(4) + 5(0) + 5(4) = 40
  3. Within-groups sum of squares Silence: (−1)²+(1)²+0²+(−2)²+(2)² = 10 Music: (−2)²+(1)²+0²+(−1)²+(2)² = 10 Café: (1)²+(−2)²+0²+(2)²+(−1)² = 10 SS_W = 10 + 10 + 10 = 30 The three groups happen to have identical spread, which is the homogeneity of variance an ANOVA wants.
  4. The table
    SourceSSdfMSF
    Between (condition)40220.008.00
    Within (error)30122.50
    Total7014
    df_B = k − 1 = 2, df_W = N − k = 12, and SS_B + SS_W = SS_Total confirms the arithmetic.
  5. Decision F(2, 12) = 8.00, F*(.05, 2, 12) = 3.89 8.00 > 3.89 → reject H₀, p = .006
  6. Effect size η² = SS_B / SS_Total = 40 / 70 = .571 57% of the variance in errors missed is associated with the sound condition. η² is biased upward in small samples; ω² (omega squared) = .483 is the less flattering estimate.
  7. Why not three t-tests Three pairwise tests at α = .05 push the chance of at least one false positive to about 14% rather than 5% (1 − .95³ = .143 if the three were independent; sharing the same data pulls it a little below that, but nowhere near back to .05). The ANOVA asks one question at one α, and the follow-up comparisons then get their own correction (Problem 14).

Sound condition affected proofreading, F(2, 12) = 8.00, p = .006, η² = .57. The omnibus test says the three means are not all equal; it does not say which pairs differ.

From §2.2 One-Way ANOVA.

14Which pairs actually differ?

Exam-level

The ANOVA in Problem 13 was significant. Now find out where the difference lives, using the same data.

Carried over from Problem 13 Means: silence 4, music 6, café 8 · n = 5 each · MS_W = 2.50, df_W = 12
  1. Compute the standard error for a pairwise comparison, and the three t statistics.
  2. Apply a Bonferroni correction and decide each comparison.
  3. Compute Tukey's HSD critical difference and check it agrees.
  4. What did the correction cost?
Worked solution
  1. Pairwise standard error: use MS_W, not just the two groups SE = √(MS_W × (1/n₁ + 1/n₂)) = √(2.50 × (1/5 + 1/5)) = √(2.50 × 0.40) = √1.00 = 1.00 Pooling all three groups' error gives df = 12 rather than the 8 a standalone pair would have.
  2. The three comparisons
    ComparisonDifferencetp
    Silence vs music22.00.069
    Silence vs café44.00.002
    Music vs café22.00.069
    Each t = difference / 1.00, evaluated on df = 12
  3. Bonferroni α_adjusted = .05 / 3 = .0167 Silence vs café: .002 < .0167 → significant The other two: .069 > .0167 → not significant
  4. Tukey's HSD as a cross-check HSD = q*(.05, k = 3, df = 12) × √(MS_W / n) = 3.77 × √(2.50 / 5) = 3.77 × 0.707 = 2.67 Any difference above 2.67 is significant. The 4-point silence-café gap clears it; the two 2-point gaps do not. Both methods reach the same verdict, with Tukey slightly less conservative, as it is built for exactly this set of all-pairs comparisons.
  5. What the correction cost Uncorrected, silence vs music at p = .069 would still not have reached .05, so nothing was lost here. In general Bonferroni trades power for protection, and with many comparisons that trade gets expensive. Tukey and Holm both keep the family-wise error rate at .05 while giving some of that power back.

Only silence against café chatter survives correction (difference = 4 errors, t(12) = 4.00, p = .002). Music sits between the two and is not distinguishable from either at α = .05.

From §2.3 Post-Hoc Tests.

15Caffeine, sleep, and the interaction

Exam-level

Forty volunteers are crossed on two factors: 200 mg caffeine or none, and a normal night's sleep or four hours. Each of the four cells holds ten people. The outcome is mean reaction time in milliseconds on a vigilance task.

Cell means (ms), n = 10 per cell
RestedDeprivedRow mean
No caffeine420520470
Caffeine410450430
Column mean415485450
Sums of squares SS_caffeine = 16,000 · SS_sleep = 49,000 · SS_interaction = 9,000 · SS_error = 36,000
  1. Complete the ANOVA table: df, MS and F for all three effects.
  2. Decide each effect at α = .05.
  3. Compute the simple effect of caffeine in each sleep condition and interpret the interaction.
  4. Report partial η² for each effect.
Worked solution
  1. Degrees of freedom df_caffeine = 2 − 1 = 1 df_sleep = 2 − 1 = 1 df_interaction = 1 × 1 = 1 df_error = N − (number of cells) = 40 − 4 = 36
  2. Error mean square MS_error = 36,000 / 36 = 1,000
  3. The table
    SourceSSdfMSFppartial η²
    Caffeine16,000116,00016.00<.001.308
    Sleep49,000149,00049.00<.001.576
    Caffeine × Sleep9,00019,0009.00.005.200
    Error36,000361,000
    partial η² = SS_effect / (SS_effect + SS_error)
  4. Decisions F*(.05, 1, 36) = 4.11 All three Fs exceed 4.11, so both main effects and the interaction are significant.
  5. Simple effects of caffeine Rested: 420 − 410 = 10 ms Deprived: 520 − 450 = 70 ms
  6. Interpret the interaction Caffeine does something quite different depending on sleep. It buys a rested person 10 ms, which is close to nothing, and a sleep-deprived person 70 ms. Reporting the caffeine main effect on its own ("caffeine improved reaction time by 40 ms") would describe an average that applies to neither group well. With a significant interaction, lead with the simple effects and treat the main effects as background.

Both main effects and the interaction are significant. Sleep deprivation is the larger influence (partial η² = .58), and caffeine helps mainly when you are short of sleep: 70 ms recovered when deprived against 10 ms when rested.

From §2.4 Factorial ANOVA (Two-Way).

16When Mauchly complains

Stretch

Twelve typists each try three keyboard layouts in a counterbalanced order. The outcome is words per minute. The software prints a repeated-measures ANOVA and a sphericity test alongside it.

Output
  • SS_conditions = 84.5, SS_error = 159.2
  • n = 12 participants, k = 3 conditions
  • Mauchly's W = .62, p = .03
  • Greenhouse–Geisser ε = .71
  1. Compute the uncorrected F and its degrees of freedom.
  2. What does Mauchly's test say, and why does it matter?
  3. Apply the Greenhouse–Geisser correction and re-decide.
  4. Report partial η².
Worked solution
  1. Degrees of freedom df_conditions = k − 1 = 2 df_error = (n − 1)(k − 1) = 11 × 2 = 22 The error term is smaller than a between-subjects design's would be, because each typist serves as their own control.
  2. Mean squares and F MS_conditions = 84.5 / 2 = 42.25 MS_error = 159.2 / 22 = 7.236 F = 42.25 / 7.236 = 5.84 F*(.05, 2, 22) = 3.44 → significant, p = .009
  3. What Mauchly's test found Sphericity means the variances of the differences between every pair of conditions are equal. Mauchly's p = .03 rejects that assumption. When sphericity fails, the F ratio is positively biased: the real Type I error rate runs above the nominal 5%, so the uncorrected p of .009 is too generous.
  4. Greenhouse–Geisser correction Multiply both df by ε = .71 df_conditions = 2 × .71 = 1.42 df_error = 22 × .71 = 15.62 F itself is unchanged at 5.84; only the reference distribution moves, and it moves against you.
  5. Re-decide F(1.42, 15.62) = 5.84, p = .020 Still below .05, so the conclusion survives. The p-value roughly doubled, which is what the correction costs. An uncorrected p near .04 would have flipped.
  6. Effect size partial η² = 84.5 / (84.5 + 159.2) = 84.5 / 243.7 = .347

Layout affected typing speed, F(1.42, 15.62) = 5.84, p = .020, partial η² = .35, with Greenhouse–Geisser applied because Mauchly's test rejected sphericity. Report the corrected df with decimals; they are meant to look like that.

From §2.5 Repeated Measures ANOVA.

17Two clinics, two very different spreads

Exam-level

A service evaluation compares how many therapy sessions clients attend at two clinics. Clinic A is small and runs a fixed 12-session program. Clinic B is large and open-ended, and its distribution is heavily right-skewed with a handful of clients attending forty or more sessions. Levene's test returns p = .004.

Given
  • Clinic A: n = 15, M = 8.2, s = 3.1
  • Clinic B: n = 45, M = 6.4, s = 6.9
  • Levene's test: p = .004
  1. Which assumption has failed, and what does Levene's p = .004 mean?
  2. The t-test assumes normality. Of what, exactly?
  3. Run Welch's t-test, including its degrees of freedom.
  4. A colleague suggests Mann–Whitney instead. Argue both sides.
Worked solution
  1. The failed assumption s²_B / s²_A = 47.61 / 9.61 = 4.95 Clinic B's variance is roughly five times Clinic A's. Levene's p = .004 says a variance gap this large would be rare if the population variances were equal, so homogeneity of variance is rejected. Note that Levene's test is itself sensitive to sample size; with unequal n and a visible five-fold gap, the plot and the ratio agree with it here.
  2. Normality of what Of the sampling distribution of each group mean, not of the raw session counts. With n = 45 the central limit theorem largely covers Clinic B despite the skew. Clinic A's n = 15 is the more fragile half, and it is the one with the tidier distribution.
  3. Welch's t — no pooling SE = √(s₁²/n₁ + s₂²/n₂) s₁²/n₁ = 9.61 / 15 = 0.6407 s₂²/n₂ = 47.61 / 45 = 1.0580 SE = √1.6987 = 1.3033 t = (8.2 − 6.4) / 1.3033 = 1.8 / 1.3033 = 1.381
  4. Welch–Satterthwaite degrees of freedom df = (s₁²/n₁ + s₂²/n₂)² / [ (s₁²/n₁)²/(n₁−1) + (s₂²/n₂)²/(n₂−1) ] = (1.6987)² / [ 0.6407²/14 + 1.0580²/44 ] = 2.8855 / [ 0.029318 + 0.025442 ] = 2.8855 / 0.054760 df = 52.69 t*(.025, 52.69) = 2.006 → 1.381 < 2.006, p = .173
  5. What pooling would have done The pooled SD is 6.20, dominated by the larger, wider Clinic B. Student's t would give 0.97 on df = 58, p = .334. Both tests fail to reject, but they disagree substantially about how far apart the clinics are, which is why the choice has to be made before looking at which answer you prefer.
  6. Welch or Mann–Whitney For Mann–Whitney: session counts are bounded below at zero and badly skewed, and the median is arguably the more meaningful summary for a service report. Against: Mann–Whitney tests stochastic dominance rather than means, and with unequal spreads its interpretation is no cleaner than Welch's. Welch handles unequal variances directly and keeps the outcome on its natural scale. Running one, reporting it, and saying why is better practice than running both and reporting the smaller p.

Homogeneity of variance fails. Welch's t(52.69) = 1.38, p = .173: the clinics do not differ detectably in sessions attended, and Welch is the defensible default whenever the variances are unequal.

From §2.6 ANOVA & t-Test Assumptions · the assumption-checks poster lists the fix for each violation.

18Five dogs each, no normality in sight

Exam-level

Two rescue shelters record days-to-adoption for five dogs each. The samples are far too small to judge normality, and one shelter has a dog that waited a month.

Days to adoption
  • Shelter A: 12, 15, 18, 22, 31
  • Shelter B: 9, 11, 14, 17, 20
  1. Rank all ten values and compute the rank sums.
  2. Compute U for each group and check them against each other.
  3. Decide at α = .05, two-tailed, given U* = 2.
  4. What is the smallest p-value this design could ever produce?
Worked solution
  1. Pool and rank, smallest first
    Value9111214151718202231
    ShelterBBABABABAA
    Rank12345678910
    No ties, so no shared ranks to average.
  2. Rank sums R_A = 3 + 5 + 7 + 9 + 10 = 34 R_B = 1 + 2 + 4 + 6 + 8 = 21 Check: 34 + 21 = 55 = N(N+1)/2 = 10(11)/2 ✓
  3. Convert rank sums to U U = R − n(n+1)/2 U_A = 34 − 5(6)/2 = 34 − 15 = 19 U_B = 21 − 15 = 6 Check: U_A + U_B = 25 = n_A × n_B ✓
  4. Decision U = min(19, 6) = 6 U*(.05, n₁ = n₂ = 5, two-tailed) = 2 6 > 2 → fail to reject H₀ Mann–Whitney is one of the few tests where you reject when the statistic is small. The exact two-tailed p here is .222.
  5. The floor on p Five against five gives C(10,5) = 252 possible arrangements. Exactly one of them puts every Shelter B dog ahead of every Shelter A dog, and one puts the reverse, so the smallest attainable two-tailed p is 2/252 = .0079. The rejection region at α = .05 is U ≤ 2, which is four arrangements in each tail out of 126, so a design this small can only detect something close to complete separation.

U = 6, p = .222: no detectable difference in days to adoption. Shelter B looks faster in every summary you could compute, and with five dogs each the test cannot tell that apart from chance.

From §2.1 Non-Parametric Alternatives.

19How first-years and finalists revise

Exam-level

A department asks 180 students how they prefer to revise. It wants to know whether preference depends on year group.

Observed counts
AloneIn pairsIn groupsTotal
First year304545120
Final year30151560
Total606060180
  1. Compute the expected counts and check the assumption about them.
  2. Compute χ² and its degrees of freedom.
  3. Decide at α = .05.
  4. Report Cramér's V and describe where the association actually sits.
Worked solution
  1. Expected counts E = (row total × column total) / N First year: 120 × 60 / 180 = 40 (all three columns) Final year: 60 × 60 / 180 = 20 (all three columns) All expected counts are 20 or above, comfortably clear of the "at least 5" rule, so no correction or exact test is needed.
  2. Cell contributions χ² = Σ (O − E)² / E (30−40)²/40 = 100/40 = 2.500 (45−40)²/40 = 25/40 = 0.625 (twice) (30−20)²/20 = 100/20 = 5.000 (15−20)²/20 = 25/20 = 1.250 (twice)
  3. Total and df χ² = 2.500 + 0.625 + 0.625 + 5.000 + 1.250 + 1.250 = 11.25 df = (rows − 1)(columns − 1) = (2 − 1)(3 − 1) = 2
  4. Decision χ²*(.05, 2) = 5.99 11.25 > 5.99 → reject H₀, p = .004
  5. Effect size Cramér's V = √( χ² / (N × min(r−1, c−1)) ) = √( 11.25 / (180 × 1) ) = √0.0625 = .25 A small-to-moderate association.
  6. Where the association lives The single largest contribution is the final-year "alone" cell at 5.00, where 30 students chose solitary revision against 20 expected. Only a quarter of first-years revise alone, and three-quarters work with someone; among finalists it is half and half. Reading the biggest cell contributions is how you turn one significant χ² into a sentence about people.

Revision preference depends on year group, χ²(2, N = 180) = 11.25, p = .004, Cramér's V = .25. Final-years revise alone far more than first-years do.

From §1.16 Chi-Square Tests · check the critical value in Statistical Tables.

20Sleep and next-day mood

Exam-level

Six volunteers keep a one-week diary. For each person the researcher records average nightly sleep in hours and average next-day mood on a 1–10 scale.

Data, n = 6
Sleep (x)4568910
Mood (y)354576
  1. Compute Pearson's r.
  2. Test it at α = .05, two-tailed.
  3. Interpret r².
  4. A magazine reports "sleep improves mood". Give two explanations of this correlation that do not involve sleep causing mood.
Worked solution
  1. Means x̄ = 42 / 6 = 7.0 ȳ = 30 / 6 = 5.0
  2. Deviations and cross-products
    xyx−x̄y−ȳ(x−x̄)²(y−ȳ)²product
    43−3−2946
    55−20400
    64−1−1111
    8510100
    9722444
    10631913
    Sums281014
  3. Pearson's r r = Σ(x−x̄)(y−ȳ) / √( Σ(x−x̄)² × Σ(y−ȳ)² ) = 14 / √(28 × 10) = 14 / √280 = 14 / 16.733 r = .837
  4. Significance test t = r√(n−2) / √(1 − r²), df = n − 2 = 4 = .837 × √4 / √(1 − .70) = 1.673 / √0.30 = 1.673 / 0.548 = 3.055 t*(.025, 4) = 2.776 → 3.055 > 2.776, reject H₀, p = .038 Significant, but only just, and on six people. A correlation of .84 needs to be enormous to clear the bar at n = 6.
  5. r² = .837² = .70 70% of the variance in mood is shared with sleep in this sample. "Shared", not "caused by", and in a sample this small the estimate is very unstable.
  6. Two non-causal readings Reverse direction: low mood disrupts sleep, which is well documented, so mood could be driving sleep rather than the other way round. Third variable: workload, pain, a new baby, or alcohol would each push sleep down and mood down at the same time, producing the correlation with no arrow between the two measured variables at all. Only random assignment to a sleep condition would separate these.

r(4) = .84, p = .038, r² = .70. Strong and significant in this sample, and entirely consistent with mood driving sleep or with something else driving both.

From §1.17 Correlation · try the correlation calculator.

21The same six people, as a line

Exam-level

Take the sleep-and-mood data from Problem 20 and fit a regression line predicting mood from sleep.

Carried over from Problem 20 x̄ = 7.0, ȳ = 5.0, Σ(x−x̄)² = 28, Σ(y−ȳ)² = 10, Σ(x−x̄)(y−ȳ) = 14
  1. Find the slope and intercept, and write the regression equation.
  2. Interpret the slope in the units of the study.
  3. Predict mood for someone sleeping 6 hours, and compute that person's residual.
  4. The researcher predicts mood at 2 hours of sleep. Why should they not?
Worked solution
  1. Slope b = Σ(x−x̄)(y−ȳ) / Σ(x−x̄)² = 14 / 28 = 0.50
  2. Intercept: the line goes through (x̄, ȳ) a = ȳ − b·x̄ = 5.0 − 0.50(7.0) = 5.0 − 3.5 = 1.50 ŷ = 1.50 + 0.50x
  3. Interpret the slope Each extra hour of sleep is associated with half a point more mood on the 1–10 scale. The intercept of 1.50 is the predicted mood at zero hours of sleep, which is outside the data and not meaningful on its own.
  4. Prediction and residual at x = 6 ŷ = 1.50 + 0.50(6) = 4.50 observed y = 4 residual = y − ŷ = 4 − 4.50 = −0.50 This person's mood was half a point below what their sleep predicted.
  5. All six residuals, as a check −0.5, +1.0, −0.5, −0.5, +1.0, −0.5 Σ residuals = 0 ✓ (always true for a least-squares fit with an intercept) SS_residual = 0.25 + 1 + 0.25 + 0.25 + 1 + 0.25 = 3.00
  6. Tie it back to r² SS_total = 10, SS_residual = 3.00, SS_regression = 7.00 r² = 7.00 / 10 = .70 ✓ matches Problem 20 s_e = √(SS_residual / (n − 2)) = √(3/4) = 0.87
  7. Why not predict at 2 hours The data run from 4 to 10 hours. Predicting at 2 assumes the straight line keeps going where nobody was measured, and there is no evidence in these six people that it does. Mood is bounded at 1, and the true relationship almost certainly bends at the extremes. Extrapolation is the most common way a correct regression produces a wrong number.

ŷ = 1.50 + 0.50x. At 6 hours the model predicts 4.50, the person scored 4, so the residual is −0.50. The fit explains 70% of the variance in mood over the observed range of 4 to 10 hours, and says nothing about 2.

From §1.18 Simple Linear Regression · fit it in the correlation & regression calculator.

22Reading the diagnostics before the coefficients

Stretch

An estate agency fits a regression predicting house price from floor area and number of bedrooms, on 40 properties. Before interpreting anything, the analyst prints the diagnostics.

Diagnostic output
  • n = 40, two predictors
  • Residuals against fitted values: a clear fan, narrow at low predicted prices and wide at high ones
  • Case 31: leverage h = 0.31, Cook's D = 0.68
  • Correlation between floor area and bedrooms: r = .92
  1. Name the problem the fan-shaped residual plot shows, and say what it damages.
  2. Case 31: is its leverage unusual? Is its Cook's D?
  3. Compute the VIF for the two predictors and interpret it.
  4. What should the analyst do before reporting coefficients?
Worked solution
  1. The fan Heteroscedasticity: residual variance grows with the fitted value, which is typical for prices. The coefficients stay unbiased, so the slope estimates are still fine on average. What breaks is everything built on the standard errors: the t-tests, the p-values and the confidence intervals are all wrong, usually too narrow at the top end.
  2. Leverage Cutoff = 2(k + 1)/n = 2(2 + 1)/40 = 6/40 = 0.15 h = 0.31, about twice the cutoff Case 31 sits far out on the predictors: a very large house, or an odd area-to-bedrooms combination. High leverage alone is not a crime, it only means the case has the opportunity to pull the line.
  3. Cook's D Common cutoffs: D > 1, or the stricter 4/n = 4/40 = 0.10 D = 0.68 → below 1, but nearly seven times 4/n High leverage plus a large Cook's D means this case is not merely unusual, it is actively moving the fitted coefficients. Refit without it and see how much the slopes shift.
  4. Multicollinearity VIF = 1 / (1 − r²) = 1 / (1 − .92²) = 1 / (1 − .8464) = 1 / .1536 = 6.51 Above the usual warning level of 5. Floor area and bedroom count carry nearly the same information, so the model cannot tell their contributions apart. The symptom to expect is a significant overall F with two unstable, possibly non-significant individual coefficients, one of which may even carry the wrong sign.
  5. What to do, in order Investigate case 31 as a data point before deleting anything, since a genuine mansion is information rather than error, and report the fit with and without it if it matters. For the fan, model log(price) instead of price, or keep the model and use heteroscedasticity-robust standard errors. For the VIF, drop one predictor or combine the two into a single size measure, and be explicit that the model now answers a narrower question. Report the diagnostics you ran, including the ones that came back clean.

Three problems, three different fixes: heteroscedasticity invalidates the standard errors, case 31 has both the leverage and the influence to bend the line, and a VIF of 6.51 means the two predictors are nearly the same variable. None of them is a reason to abandon the model, and all three change what the coefficients mean.

From §2.8 Regression Diagnostics and §2.13 Assumptions of Regression.

23A table from three lines of output

Exam-level

A sports scientist compares three warm-up routines on how far athletes can reach in a sit-and-reach test, eight athletes per routine. The paper reports only the descriptive statistics, so the ANOVA table has to be rebuilt from them.

Descriptives, centimeters
  • Static: n = 8, M = 12.0, SD = 3.2
  • Dynamic: n = 8, M = 15.0, SD = 2.9
  • Ballistic: n = 8, M = 18.0, SD = 3.5
  1. Find the grand mean, then SSG, SSE and SST.
  2. Fill in the three degrees of freedom and both mean squares.
  3. Compute F and decide at α = .05 against the critical value 3.47.
  4. Report η² and the pooled standard deviation.
Worked solution
  1. Grand mean Equal group sizes, so it is the mean of the three means x̄ = (12.0 + 15.0 + 18.0) / 3 = 15.0 cm With unequal group sizes this shortcut fails and the grand mean has to be the weighted average Σnᵢx̄ᵢ / N.
  2. Between groups SSG = Σ nᵢ(x̄ᵢ − x̄)² = 8(12 − 15)² + 8(15 − 15)² + 8(18 − 15)² = 8(9) + 8(0) + 8(9) = 144.00
  3. Within groups, from the SDs SSE = Σ (nᵢ − 1)sᵢ² = 7(3.2² + 2.9² + 3.5²) = 7(10.24 + 8.41 + 12.25) = 7(30.90) = 216.30 This is the form to reach for whenever a question hands you standard deviations instead of raw scores. It works because each group's SD is already built on nᵢ − 1, so multiplying by that df undoes the division and returns the group's own sum of squares.
  4. Total, and the check SST = 144.00 + 216.30 = 360.30 Nothing else was available to compute SST from here, since the raw scores are gone. When you do have them, computing all three separately and seeing them add up is the cheapest error check there is.
  5. Degrees of freedom and mean squares dfG = I − 1 = 2 dfE = N − I = 24 − 3 = 21 dfT = 23 MSG = 144.00 / 2 = 72.00 MSE = 216.30 / 21 = 10.30
  6. The F ratio F = 72.00 / 10.30 = 6.99 on (2, 21) 6.99 > 3.47, so reject H₀ at α = .05 (exact p = .005)
  7. Effect size and pooled SD η² = 144.00 / 360.30 = .40 sp = √10.30 = 3.21 cm Two of these numbers do different jobs. η² says the routine accounts for about 40% of the variation in reach. sp is the error term any follow-up comparison between two routines will use, and it is close to the individual group SDs, which is what you would hope given the ANOVA assumes they are estimating the same thing.

SSG = 144.00 on 2 df, SSE = 216.30 on 21, SST = 360.30 on 23. MSG = 72.00, MSE = 10.30, F(2, 21) = 6.99, p = .005. Warm-up routine affected reach, η² = .40, sp = 3.21 cm.

From §2.2 One-Way ANOVA. Check it in Build the ANOVA Table or in the APA formatter.

24The table with four cells missing

Warm-up

A printout has been photocopied badly and four cells of the ANOVA table are unreadable. Forty participants were assigned to four versions of an instruction leaflet, and the outcome is a comprehension score.

What survived the photocopier
  • Between groups: SS = ?, df = 3, MS = ?
  • Within groups: SS = 216.00, df = ?, MS = ?
  • Total: SS = 396.00, df = 39
  1. Recover the four missing cells and the F ratio.
  2. Decide at α = .05 against the critical value F*(3, 36) = 2.87.
  3. Report η² and the pooled standard deviation.
Worked solution
  1. The two sums of squares SST = SSG + SSE, so SSG = 396.00 − 216.00 = 180.00 The partition identity is doing all the work here. Two of the three sums of squares always determine the third.
  2. The missing df dfT = dfG + dfE, so dfE = 39 − 3 = 36 Read the design back out of the df while you are here: dfG = I − 1 = 3 means four groups, and dfT = N − 1 = 39 means forty participants. Both match the scenario, which is a check worth making every time.
  3. Mean squares MSG = 180.00 / 3 = 60.00 MSE = 216.00 / 36 = 6.00
  4. The F ratio and the decision F = 60.00 / 6.00 = 10.00 on (3, 36) 10.00 > 2.87, so reject H₀ (exact p = .00006) A common slip is to divide the sums of squares instead: 180.00 / 216.00 = 0.83, which would have said the opposite. F is a ratio of mean squares, so each sum of squares has to meet its own df first.
  5. Effect size and pooled SD η² = 180.00 / 396.00 = .45 sp = √6.00 = 2.45 The leaflet version accounts for about 45% of the variation in comprehension. What the table cannot say is which versions differ, and the pooled SD is exactly what the follow-up comparisons will need to find out.

SSG = 180.00, dfE = 36, MSG = 60.00, MSE = 6.00, F(3, 36) = 10.00, p < .001. Reject H₀: the four leaflets are not equally well understood, η² = .45, sp = 2.45.

From §2.2 One-Way ANOVA. The exam mode of Build the ANOVA Table generates more of these.

25One question, asked before the data

Exam-level

An occupational health team tests whether any kind of scheduled break improves afternoon proofreading accuracy. Forty office workers are assigned to four conditions, ten each: no break at all, three two-minute micro-breaks, one fifteen-minute break, and one fifteen-minute break spent walking outdoors. The team wrote one question into its protocol before collecting anything: does taking a break of any kind beat taking none? The outcome is the percentage of planted errors caught.

Descriptives, percent of errors caught, n = 10 per condition
  • No break: M = 52.0, SD = 6.0
  • Micro-breaks: M = 58.0, SD = 5.0
  • Long break: M = 61.0, SD = 7.0
  • Walk outdoors: M = 65.0, SD = 6.0

Critical values: F*(3, 36) = 2.87, t*(36) = 2.028.

  1. Build SSE, dfE and MSE from the standard deviations, then run the omnibus F.
  2. Write the coefficients for the team's planned question and check that they qualify as a contrast.
  3. Compute the estimate, its standard error and the t, and decide at α = .05.
  4. Give the 95% confidence interval and say what it means in percentage points.
  5. Redo the contrast with whole-number coefficients and say what changes.
Worked solution
  1. The error term, from the SDs SSE = Σ (nᵢ − 1)sᵢ² = 9(6.0² + 5.0² + 7.0² + 6.0²) = 9(36 + 25 + 49 + 36) = 9(146) = 1314.00 dfE = N − I = 40 − 4 = 36,  MSE = 1314.00 / 36 = 36.50 Everything after this borrows MSE. It is the one number the contrast needs from the ANOVA table, which is why the table is worth building even when the question was never about all four groups.
  2. The omnibus F, for the record x̄ = (52 + 58 + 61 + 65) / 4 = 59.00 SSG = 10[(−7)² + (−1)² + 2² + 6²] = 10(49 + 1 + 4 + 36) = 900.00 MSG = 900.00 / 3 = 300.00,  F = 300.00 / 36.50 = 8.22 on (3, 36) 8.22 > 2.87, so reject H₀ (exact p = .00027), η² = 900 / 2214 = .41 The equal group sizes are what let the grand mean be the plain average of the four means. A planned contrast does not actually need this test to have been significant, since it was not chosen by looking at the means, but reporting it costs one line and tells a reader the design worked.
  3. The coefficients a = [1, −⅓, −⅓, −⅓] Σaᵢ = 1 − ⅓ − ⅓ − ⅓ = 0 ✓ One group on one side, the other three averaged on the other. Coefficients of [1, −1, −1, −1] would sum to −2 and would not be a contrast at all: the three treatment weights have to share the −1 between them, not each take it.
  4. Estimate and standard error ψ̂ = 52.00 − (58.00 + 61.00 + 65.00)/3 = 52.00 − 61.33 = −9.33 Σ aᵢ²/nᵢ = (1 + ⅑ + ⅑ + ⅑) / 10 = (4/3) / 10 = .1333 SE = √(36.50 × .1333) = √4.8667 = 2.21
  5. The test t = −9.33 / 2.21 = −4.23 on df = N − I = 36 |−4.23| > 2.028, so reject H₀ (exact two-tailed p = .00015) The direction was in the protocol, so a one-tailed test at .05 would have been legitimate here and would halve that p to .000077. It changes nothing about this decision, and the write-up has to say the direction was planned.
  6. The interval −9.33 ± 2.028 × 2.21 = −9.33 ± 4.47 95% CI [−13.81, −4.86] percentage points Reading it out loud: workers given no break caught somewhere between about 5 and 14 percentage points fewer of the planted errors than the average of the three break conditions. That sentence is what the study was for, and the t alone cannot produce it.
  7. The same contrast, in whole numbers a = [3, −1, −1, −1]:  ψ̂ = 156.00 − 184.00 = −28.00 Σ aᵢ²/nᵢ = (9 + 1 + 1 + 1)/10 = 1.20,  SE = √(36.50 × 1.20) = 6.62 t = −28.00 / 6.62 = −4.23, unchanged. CI [−41.42, −14.58] Everything scaled by three: the estimate, the standard error and both interval limits. The t is a ratio of two of them, so it cannot move. Type the whole numbers into SPSS and report the fractional version, or divide the output by three yourself.

MSE = 36.50 on 36 df; F(3, 36) = 8.22, p < .001, η² = .41. The planned contrast of no break against the average of the three break conditions gives ψ̂ = −9.33, SE = 2.21, t(36) = −4.23, p < .001, 95% CI [−13.81, −4.86]. Any break beats none, by roughly 5 to 14 percentage points.

From §2.3 Post-Hoc Tests. Type the coefficients into Contrast Builder to check every line of it.

26Three formats and three comparisons

Exam-level

Seventy-five students are randomly assigned to one of three revision formats for the same material, twenty-five per format, and sit a common exam. Nobody specified which formats to compare in advance, so all three pairs are on the table afterwards. The ANOVA has already been run.

Group means and the ANOVA table
  • Format A (rereading): M = 64.0; Format B (summarizing): M = 70.0; Format C (self-testing): M = 72.0
  • Between groups: SS = 866.67, df = 2, MS = 433.33
  • Within groups: SS = 7200.00, df = 72, MS = 100.00
  • Critical values: F*(2, 72) = 3.12, t*(72) = 1.993, t**(72) at .05/3 two-sided = 2.451
  1. Confirm the omnibus decision, and give sp and the standard error every pair will share.
  2. Compute all three pairwise t statistics on the pooled error term.
  3. Decide each one twice, uncorrected and Bonferroni-corrected, and say which comparison the correction costs you.
  4. Write the results sentence in APA style, naming the correction.
Worked solution
  1. The omnibus, and the shared standard error F = 433.33 / 100.00 = 4.33 on (2, 72); 4.33 > 3.12, so reject H₀ (exact p = .017) sp = √MSE = √100.00 = 10.00 SE = 10.00 × √(1/25 + 1/25) = 10.00 × √.08 = 2.83 Equal group sizes mean one standard error covers all three pairs. With unequal n each pair would get its own, and only sp would be shared. η² = 866.67 / 8066.67 = .11, which is a modest effect for a significant omnibus and worth carrying into the write-up.
  2. Three t statistics A vs. B: t = (64.0 − 70.0) / 2.83 = −2.12 A vs. C: t = (64.0 − 72.0) / 2.83 = −2.83 B vs. C: t = (70.0 − 72.0) / 2.83 = −0.71 All three on df = N − I = 75 − 3 = 72 Every one of these uses 72 degrees of freedom, not the 48 that two groups of 25 would give on their own. That is the pooled error term earning its keep.
  3. Two verdicts each Uncorrected, against t* = 1.993: A vs. B reject, A vs. C reject, B vs. C keep Bonferroni, against t** = 2.451: A vs. B keep, A vs. C reject, B vs. C keep Exact p-values: .037, .006 and .482 The correction costs the A-versus-B comparison, whose t of 2.12 sits between the two critical values. The same decision in the other convention: multiply each p by 3 to get .112, .018 and 1.000, then read them against .05. Two roads, same three answers, and SPSS drives the second.
  4. The write-up Revision format affected exam scores, F(2, 72) = 4.33, p = .017, η² = .11. With Bonferroni-corrected pairwise comparisons on the pooled error term, self-testing outperformed rereading, Mdiff = 8.00, t(72) = 2.83, p = .018 corrected. Neither the rereading-versus-summarizing difference of 6.00 points, t(72) = 2.12, p = .112 corrected, nor the summarizing-versus-self-testing difference of 2.00 points, t(72) = 0.71, p = 1.000 corrected, reached significance. Two habits that reviewers notice. The word Bonferroni appears, because a bare p-value from a family of comparisons cannot be interpreted without it. And the corrected p-values are labeled as corrected, so nobody has to guess whether .018 was multiplied by three already.

sp = 10.00, shared SE = 2.83 on 72 df. t = −2.12, −2.83 and −0.71 for A vs. B, A vs. C and B vs. C. Uncorrected, the first two are significant; under Bonferroni only A vs. C survives, since 2.12 falls between t* = 1.993 and t** = 2.451. Corrected p-values .112, .018 and 1.000.

From §2.3 Post-Hoc Tests. The All pairs panel of Contrast Builder lays out both verdicts side by side, and the APA formatter handles the omnibus sentence.

27Two reading apps, fifteen children

Exam-level

Fifteen children are randomly assigned to learn spelling either from a paper workbook or from a tablet app, and are scored a fortnight later on words recalled. The scores are bounded below, badly skewed, and there are seven children in one group and eight in the other, so the class is told to run a Mann–Whitney U test by hand against the printed table.

Words recalled
  • Workbook (n1 = 7): 12, 15, 17, 20, 23, 26, 31
  • App (n2 = 8): 18, 22, 25, 28, 30, 33, 36, 40
  • Critical values: U*(7, 8) = 10 two-tailed at .05, 6 two-tailed at .01, 13 one-tailed at .05
  1. Rank the fifteen scores as one pooled set and give both rank sums, with the check that they are right.
  2. Compute U1 and U2, and say which of them is the test statistic.
  3. Decide the test at two-tailed α = .05.
  4. The researcher now says the app was predicted to win from the start. Does the decision change, and is she entitled to make it?
Worked solution
  1. Pool, rank, and total Sorted: 12, 15, 17, 18, 20, 22, 23, 25, 26, 28, 30, 31, 33, 36, 40 Workbook ranks: 1, 2, 3, 5, 7, 9, 12 → R1 = 39 App ranks: 4, 6, 8, 10, 11, 13, 14, 15 → R2 = 81 Check: 39 + 81 = 120 = N(N + 1)/2 = 15 × 16 / 2 Rank the pooled set, not each group separately. Nothing ties here, so every rank is a whole number; a tie would take the average of the ranks the tied values were competing for. The check on the two sums is worth thirty seconds, because a misranked value is the single commonest way this question goes wrong.
  2. Both U values U1 = n1n2 + n1(n1 + 1)/2 − R1 = 56 + 28 − 39 = 45 U2 = n1n2 + n2(n2 + 1)/2 − R2 = 56 + 36 − 81 = 11 Check: 45 + 11 = 56 = n1n2 The statistic is U = 11, the smaller of the two Working the second one out from its own rank sum rather than by subtracting from 56 costs one line and checks the ranking again. The smaller U belongs to the app, which is the group sitting higher in the pooled order, and that is what makes it the interesting one.
  3. The table, which runs the other way U = 11 against U*(7, 8) = 10 at two-tailed .05 11 > 10, so keep H₀ A U table prints a ceiling. The statistic counts how much the two groups still overlap, so it shrinks as they separate, and only a value at or below the cell counts as significant. Eleven misses by one. The exact two-tailed p is .054, which says the same thing from the other direction and shows how close this was.
  4. One tail, and who is allowed to use it One-tailed at .05: U*(7, 8) = 13, and 11 ≤ 13, so reject H₀ Exact one-tailed p = .027 The arithmetic is not in question; the timing is. A direction chosen before the data arrived is a legitimate one-tailed test and this result rejects. A direction chosen after seeing that the app came out ahead is not a test at all, it is the same two-tailed .054 relabeled, and a marker will treat it as such. If your table prints only two-tailed columns, double the α before you look: a directional test at .05 takes its cell from the .10 column, because half of a two-tailed α is reserved for the tail nobody is watching.

R1 = 39 and R2 = 81, summing to 120. U1 = 45, U2 = 11, so U = 11. Against U* = 10 at two-tailed .05 the null survives, exact p = .054. A one-tailed test predicted in advance would reject against U* = 13, exact p = .027, but only if the direction was stated first.

From §2.1 Non-Parametric Alternatives. The Mann–Whitney tab of the rank ledger prints the same arithmetic for its own data, and Table U is the table this problem reads.

28Nine sleepers, one of whom did not budge

Exam-level

Nine adults rate their sleep quality on a 50-point clinical scale before and after four weeks with a wind-down routine. Higher is better. The ratings are ordinal and the sample is small, so the analysis is a Wilcoxon signed-ranks test done by hand.

Sleep-quality scores
  • Before: 24, 31, 19, 27, 22, 35, 28, 30, 21
  • After: 29, 39, 19, 38, 25, 34, 35, 44, 23
  • Critical values: W*(8) = 3 two-tailed at .05 and 0 two-tailed at .01; W*(9) = 5 two-tailed at .05
  1. Form the nine differences and say how many pairs actually enter the test.
  2. Rank the surviving differences by absolute size and give T+ and T, with the check.
  3. Decide the test at two-tailed α = .05 and again at .01.
  4. A classmate keeps the unchanged participant, arguing that discarding data is never right. What does that do to the test?
Worked solution
  1. Differences, and one that is not a difference after − before: +5, +8, 0, +11, +3, −1, +7, +14, +2 The third participant scored 19 both times, so that pair is dropped n = 8 A pair with a difference of exactly zero has no direction, and the whole test is built on attaching a sign to a rank. There is no honest sign to give it, so it goes, and n becomes the number of pairs left. This is the step most often skipped, and it changes both the ranking and the row of the table you read.
  2. Rank the sizes, then sort by sign |d| sorted: 1, 2, 3, 5, 7, 8, 11, 14 → ranks 1 to 8 +5 → 4, +8 → 6, +11 → 7, +3 → 3, −1 → 1, +7 → 5, +14 → 8, +2 → 2 T+ = 4 + 6 + 7 + 3 + 5 + 8 + 2 = 35 T = 1 Check: 35 + 1 = 36 = n(n + 1)/2 = 8 × 9 / 2 Rank the absolute sizes, ignoring sign, and only then look back at which sign each rank came from. The one person who got worse moved least, so the negative side collects a single rank of 1. Had two differences tied in size they would share the average of their two ranks, and the sums would still have to reach 36.
  3. Both decisions W = min(35, 1) = 1 At .05: 1 ≤ 3, so reject H₀ At .01: 1 > 0, so keep H₀ Exact two-tailed p = .016 Like U, W is a ceiling, and for the same reason: it counts the ranks piled on the losing side, so agreement drives it down. The .01 cell of 0 means that with only eight pairs the evidence has to be perfect, every single person moving the same way, before the test can reject at that level. The exact p of .016 sits between the two, which is exactly what the two decisions say.
  4. Keeping the zero Two things break at once. The unchanged participant has to be given a signed rank, and there is no sign to give, so whichever side receives it is an invention. And n becomes 9, which sends you to a different row of the table, where the cell is 5 rather than 3. Here the verdict happens to survive both errors, which is the worst kind of luck: the procedure was wrong and the answer came out right anyway. The honest reading of a dropped pair is not that data was discarded but that this participant contributes nothing to a question about direction, and the smaller n reports that loss of information rather than hiding it.

One zero difference is dropped, so n = 8. T+ = 35 and T = 1, summing to 36. W = 1, which rejects at two-tailed .05 against W* = 3 but not at .01 against W* = 0. Exact two-tailed p = .016.

From §2.1 Non-Parametric Alternatives. The Wilcoxon tab of the rank ledger shows the same ledger for its own seven pairs, and Table W holds both cells.

29Exercise and wellbeing, landing on the cell

Exam-level

Eight adults report their typical daily exercise in minutes and complete a wellbeing questionnaire. Exercise minutes are heavily skewed by two people who train far more than the rest, so the relationship is described with Spearman's ρs rather than Pearson's r.

Minutes and wellbeing scores
  • Exercise (min/day): 10, 15, 20, 25, 30, 40, 50, 65
  • Wellbeing: 52, 41, 44, 47, 63, 55, 70, 58
  • Critical values: ρs*(8) = .738 two-tailed at .05, .881 two-tailed at .01, .643 one-tailed at .05
  1. Rank both columns, build the d and d² columns, and give Σd².
  2. Compute ρs from the shortcut.
  3. Decide at two-tailed α = .05 and at .01, and say carefully why the first decision goes the way it does.
  4. Two of the wellbeing scores turn out to have been transcribed wrongly and are in fact equal. What has to change?
Worked solution
  1. Two columns of ranks Exercise is already in order, so rank x = 1, 2, 3, 4, 5, 6, 7, 8 Wellbeing sorted: 41, 44, 47, 52, 55, 58, 63, 70 rank y in the original order = 4, 1, 2, 3, 7, 5, 8, 6 d = −3, 1, 1, 1, −2, 1, −1, 2 d² = 9, 1, 1, 1, 4, 1, 1, 4 → Σd² = 22 Rank the two columns separately, then subtract row by row. The signs of d do not matter once you square, but they are worth writing down anyway: their sum has to be zero, which catches a misranked row before it reaches the formula.
  2. The shortcut ρs = 1 − 6Σd² / (n(n² − 1)) = 1 − 6(22) / (8 × 63) = 1 − 132 / 504 = 1 − .2619 = .738 Pearson's r on the raw numbers would give .656. The two heavy exercisers stretch the x axis and flatten the linear fit, while the ranks treat them simply as the sixth, seventh and eighth largest. That gap is the reason for choosing ρs here, and it is worth a sentence in any write-up.
  3. Two decisions, and one boundary At two-tailed .05: .738 against a cell of .738 The rule is reject when |ρs| REACHES the critical value, so reject H₀ At two-tailed .01: .738 < .881, so keep H₀ Exact two-tailed p = .046 Landing exactly on the cell is not a coincidence to be embarrassed about; it is what a discrete statistic does. Σd² = 22 is the very value that defines the .05 cell for n = 8, and the exact two-tailed probability of a result this extreme is .046, comfortably under .05. Had the rule been a strict inequality, this genuinely significant result would have been thrown away. Spearman's table is a floor rather than a ceiling, because ρs is a correlation and grows with the strength of the relationship, unlike U and W.
  4. What a tie costs The two tied scores share the average of the two ranks they were competing for, and the y column stops being the whole numbers 1 to 8. The shortcut was derived on the assumption that both columns are exactly those whole numbers, so it is no longer algebraically exact and will drift a little from the truth. The fix is to go back to the definition and run Pearson's formula on the two rank columns, which is what Spearman's ρs has always been and what SPSS and JASP compute whether or not anything ties. With one tied pair in eight the drift is small, but "small" is not a number you can quote.

Σd² = 22, so ρs = 1 − 132/504 = .738. That reaches the two-tailed .05 cell of .738 exactly, so the null is rejected (exact p = .046), but it falls short of the .01 cell of .881. Ties would break the shortcut and force Pearson's formula on the ranks.

From §2.1 Non-Parametric Alternatives and §1.17 Correlation. The Spearman tab of the rank ledger builds the same d² column live, and Table ρ holds both cells.

30Four cells, and everything that falls out of them

Exam-level

Forty students are each given the same two-hour assignment and randomly assigned to one of four conditions: they work either in a quiet reading room or in a busy café, and their phone is either left in another room or sitting on the desk. The outcome is the number of minutes of focused work recorded by a timer that stops when they switch away from the document. Ten students land in each cell.

Cell means, in minutes
  • Quiet room, phone in another room: 84  ·  quiet room, phone on the desk: 64
  • Busy café, phone in another room: 62  ·  busy café, phone on the desk: 58
  • Call the place factor A (A₁ quiet, A₂ café) and the phone factor B (B₁ away, B₂ on the desk)
  1. Lay the four means out as a 2 × 2 and fill in both row margins, both column margins and the grand mean.
  2. Give the main effect of each factor, and check each one against its own two simple main effects.
  3. Give all four simple main effects and the difference of differences. Is the interaction spreading or crossover?
  4. Describe the interaction plot with A on the horizontal axis and one line per level of B.
  5. A colleague summarizes the study as "leaving your phone in another room buys you twelve minutes". Say what is right and what is misleading about that.
Worked solution
  1. The square, with its margins Row A₁: 84 and 64  →  margin (84 + 64)/2 = 74 Row A₂: 62 and 58  →  margin (62 + 58)/2 = 60 Column B₁: 84 and 62  →  margin (84 + 62)/2 = 73 Column B₂: 64 and 58  →  margin (64 + 58)/2 = 61 Grand mean = (84 + 64 + 62 + 58)/4 = 268/4 = 67 Two arithmetic checks come free. The two row margins average to the grand mean, (74 + 60)/2 = 67, and so do the two column margins, (73 + 61)/2 = 67. Both hold only because every cell has the same ten students in it; with unequal cells a margin is a weighted average and these tidy identities go away.
  2. Two main effects Main effect of A = 60 − 74 = −14 minutes (the café costs 14) Main effect of B = 73 − 61 = +12 minutes (the phone away buys 12) Each is a statement about an average over the other factor. The café figure averages over the two phone conditions; the phone figure averages over the two places.
  3. Four simple main effects, and their difference A at B₁: 62 − 84 = −22    A at B₂: 58 − 64 = −6 Mean of the two: (−22 + −6)/2 = −14, which is the main effect of A B at A₁: 84 − 64 = +20    B at A₂: 62 − 58 = +4 Mean of the two: (20 + 4)/2 = +12, which is the main effect of B Difference of differences = −6 − (−22) = +16 Difference the other factor first and you get 4 − 20 = −16, the same size with the other sign. There is only one interaction here, and which sign it wears depends on which factor you subtracted first. Both simple main effects of A point the same way and both simple main effects of B point the same way, so this is a spreading interaction rather than a crossover: the phone matters in both places, five times as much in the quiet room.
  4. The picture Two points on the left at A₁, 84 for B₁ and 64 for B₂; two on the right at A₂, 62 and 58. The B₁ line falls steeply from 84 to 62; the B₂ line drops gently from 64 to 58. The B₁ line stays above the B₂ line across the whole plot, so the lines do not cross inside the range, and their different slopes are the interaction. A 20-minute gap on the left has narrowed to a 4-minute gap on the right.
  5. What the summary gets wrong Twelve minutes is the correct main effect of B, so the number is not wrong. What is misleading is the implied "wherever you work". The phone bought 20 minutes in the quiet room and 4 in the café, and 12 is the average of two numbers, neither of which is 12. With an interaction this size the honest summary reports it first and then gives the two simple main effects: leaving the phone in another room helped in both settings, substantially in the quiet room and barely in the café.

Margins 74 and 60 down the rows, 73 and 61 across the columns, grand mean 67. Main effects −14 for place and +12 for the phone. Simple main effects −22 and −6 for place, +20 and +4 for the phone, each pair averaging to its own main effect; the difference of differences is 16, a spreading interaction. The twelve-minute claim is the right average and the wrong story.

From §2.4 Factorial ANOVA. Type the four cell means into the Interaction Plot Explorer and every number above is printed for you.

31The two-way table with four cells missing

Exam-level

Students take notes on the same lecture either on a laptop or in longhand, and are quizzed either the same day, one week later or three weeks later. Every one of the six conditions holds the same number of students. The output below arrived with four cells missing.

Two-way ANOVA, quiz score out of 40
  • Medium (A): SS = 96.00, df = 1, MS = 96.00, F = ?
  • Delay (B): SS = 150.00, df = 2, MS = ?, F = ?
  • A × B: SS = 88.00, df = 2, MS = 44.00, F = ?
  • Error: SS = ?, df = ?, MS = ?
  • Total: SS = 838.00, df = 47
  • Critical values: F*(1, 42) = 4.07 and F*(2, 42) = 3.22, both at α = .05
  1. How many levels does each factor have, how many students were there in total, and how many sat in each cell? Answer from the degrees of freedom alone.
  2. Complete the error row and the three F ratios.
  3. Decide each of the three tests at α = .05, and give partial η² for the interaction.
  4. Which line do you report first, and what does that decision do to the two sentences about the main effects?
Worked solution
  1. Reading the design off the df column dfA = I − 1 = 1  →  I = 2 levels of medium dfB = J − 1 = 2  →  J = 3 delays, and (I − 1)(J − 1) = 2 checks out dfT = N − 1 = 47  →  N = 48 students IJ = 6 cells, so n = 48/6 = 8 per cell A two-way table carries its own design in the df column, which is why an exam can hand you nothing else and still expect the sample size back. The same trick works in reverse: quote F(2, 42) in a paper and a reader can reconstruct three levels and 48 people.
  2. The error row, then the ratios SSE = SST − SSA − SSB − SSAB = 838 − 96 − 150 − 88 = 504.00 dfE = NIJ = 48 − 6 = 42, and 1 + 2 + 2 + 42 = 47 as it must MSE = 504/42 = 12.00 MSB = 150/2 = 75.00 FA = 96/12 = 8.00    FB = 75/12 = 6.25    FAB = 44/12 = 3.67 All three F ratios divide by the same MSE, which is the only reason the missing error row had to be recovered first. Its square root, sp = √12 = 3.46 points, is the residual spread worth quoting beside the means.
  3. Three decisions and an effect size Medium: 8.00 > 4.07, so reject (exact p = .007) Delay: 6.25 > 3.22, so reject (exact p = .004) A × B: 3.67 > 3.22, so reject (exact p = .034) ηp² for the interaction = 88 / (88 + 504) = 88/592 = .149 Partial η² puts each effect over itself plus the error rather than over the total, so the three values here (.160, .229 and .149) are not shares of one pie and do not add to anything. The interaction is the smallest of the three and still comfortably past its critical value, because 42 error degrees of freedom buy a good deal of precision.
  4. The reading order The interaction line goes first, because it is significant and it changes what the two lines above it are allowed to say. The medium effect of 8.00 and the delay effect of 6.25 are both real, but each is an average over the other factor, and a significant A × B says the medium's advantage is not the same at every delay. The write-up leads with the interaction, then reports each main effect as qualified by it, and then tests the simple main effects that say what actually happened: the medium's effect at each of the three delays, or the delay's effect within each medium. Reporting the two main effects first and mentioning the interaction as an afterthought inverts the logic of the table.

Two levels of medium, three delays, 48 students, 8 per cell. SSE = 504.00 on 42 df, MSE = 12.00; MSB = 75.00; F(1, 42) = 8.00, F(2, 42) = 6.25 and F(2, 42) = 3.67, all three significant at .05, with ηp² = .149 for the interaction. Report the interaction first and treat both main effects as qualified by it.

From §2.4 Factorial ANOVA and §2.2 One-Way ANOVA. The printed F tables hold both criticals, and Where the Variance Goes shows what the error row would have swallowed had the delay factor been left out.

📐 Regression · Models with several predictors

Eight problems on models with several predictors, most of them taught in Stats 2. The arithmetic here is lighter than in Stats 1, and the judgment is heavier: reading a coefficient correctly, deciding what a model is allowed to claim, and noticing when the output is warning you about itself.

23Two predictors, two scales

Exam-level

A study of 120 first-year students regresses final exam score on weekly study hours and average nightly sleep. A journalist reads the output and writes that sleep matters two and a half times more than studying, because its coefficient is bigger.

Regression output and descriptives
  • Model: exam = 29.57 + 1.30(study hours) + 3.20(sleep hours)
  • SE = 0.24 for study, 0.80 for sleep; both p < .001, df = 117
  • R² = .34, F(2, 117) = 30.14, p < .001
  • Exam score: M = 64.0, SD = 12.4
  • Study hours: M = 9.5, SD = 4.0
  • Sleep hours: M = 6.9, SD = 1.2
  1. Put the study-hours coefficient into a sentence a non-statistician would understand.
  2. Compute the standardized coefficients. Is the journalist right?
  3. Predict the exam score for a student who studies 10 hours a week and sleeps 7 hours a night.
  4. Compute adjusted R², and say why the intercept should not be interpreted.
Worked solution
  1. Reading a raw coefficient Among students who sleep the same amount, each extra hour of weekly study is associated with 1.30 more exam points. The phrase doing the work is "who sleep the same amount": every coefficient in a multiple regression is the effect of its predictor with the others held fixed. Nothing here licenses the word causes, since nobody was assigned their study hours.
  2. Standardizing β = b × (SD of the predictor) / (SD of the outcome) βstudy = 1.30 × 4.0 / 12.4 = 5.20 / 12.4 = .419 βsleep = 3.20 × 1.2 / 12.4 = 3.84 / 12.4 = .310 The journalist has it backwards. Study hours vary far more between students (SD = 4.0 hours) than sleep does (SD = 1.2 hours), so an hour of study is a much smaller step through the actual data than an hour of sleep. Once both are expressed in standard deviations, studying is the stronger predictor. Raw coefficients are only comparable when the predictors share a scale, and hours-per-week and hours-per-night do not.
  3. A prediction ŷ = 29.57 + 1.30(10) + 3.20(7) = 29.57 + 13.00 + 22.40 = 64.97 ≈ 65.0 points Both values sit near the middle of their observed ranges, which is the only region where a prediction is safe.
  4. Adjusted R² and the intercept adj R² = 1 − (1 − R²)(n − 1)/(n − k − 1) = 1 − (.66)(119/117) = 1 − .671 = .329 ≈ .33 Barely below R², because two predictors is a cheap model at n = 120. The intercept of 29.57 is the predicted score for a student who studies zero hours a week and sleeps zero hours a night. No such student exists in the data, and the model has no information about that corner of the space, so the number is an artifact of where the plane happens to cross the axis.

βstudy = .42 against βsleep = .31, so studying is the stronger predictor and the headline is wrong. Predicted score at 10 study hours and 7 sleep hours is 65.0, adjusted R² = .33.

From §2.9 Multiple Regression · the same distinction shows up in the effect-size converter.

24The reading program that started ahead

Exam-level

A school offers an after-school reading program. Thirty children enroll and thirty comparable children do not, and everyone sits the same reading test before and after the term. The program leader reports a 6.4-point advantage.

Test scores, 30 per group
  • Pretest: program M = 52.0 (SD = 10.5), comparison M = 45.0 (SD = 10.5)
  • Posttest: program M = 61.4 (SD = 9.0), comparison M = 55.0 (SD = 9.0)
  • Pooled within-group slope of posttest on pretest: b = 0.62
  • Independent t-test on posttest alone: t(58) = 2.75, p = .008
  • ANCOVA with pretest as covariate: F(1, 57) = 1.48, p = .229
  1. Compute each group's adjusted posttest mean.
  2. How large is the adjusted effect, and why does it differ from 6.4?
  3. The program's newsletter says the course "raises reading scores by 6.4 points." Rewrite that sentence honestly.
  4. Which ANCOVA assumption would you check first here, and what would you do if it failed?
Worked solution
  1. The covariate's grand mean X̄ = (52.0 + 45.0) / 2 = 48.5 Equal group sizes, so the simple average is the grand mean. ANCOVA compares the groups at this pretest value.
  2. Adjusted means adjusted Ȳj = Ȳj − b(X̄j − X̄) program: 61.4 − 0.62(52.0 − 48.5) = 61.4 − 2.17 = 59.23 comparison: 55.0 − 0.62(45.0 − 48.5) = 55.0 + 2.17 = 57.17 The program group is slid down because it started 3.5 points above average, and the comparison group is slid up for the mirror-image reason.
  3. The adjusted effect 59.23 − 57.17 = 2.06 points shortcut: adjusted = raw − b × (covariate gap) = 6.4 − 0.62(7.0) = 6.4 − 4.34 = 2.06 Of the 6.4-point advantage, 4.34 points were already there in September. What the term added, at least as far as this analysis can see, is 2.06 points, and F(1, 57) = 1.48, p = .229 says even that is within what chance produces at this sample size.
  4. Rewriting the claim "Children in the program finished 6.4 points ahead, but they also began the year 7.0 points ahead. Adjusting for the starting gap leaves a 2.06-point difference that is not statistically distinguishable from zero, F(1, 57) = 1.48, p = .229." Note also that families chose the program, so even the adjusted figure is an observational comparison. Pretest score is the one baseline difference that was measured; motivation, home support and reading habits were not.
  5. Homogeneity of regression slopes ANCOVA fits one slope to both groups and reports a single adjusted gap. That gap only exists if the pretest predicts the posttest equally in each group. Test it by adding the group × pretest interaction: if the interaction is significant, the lines are not parallel and there is no single number to report. You would then fit the interaction model and describe how the program's effect changes across the pretest range, which is often the more interesting finding anyway.

Adjusted means 59.23 and 57.17, an adjusted difference of 2.06 points against a raw difference of 6.4. Two thirds of the headline gap was baseline imbalance, and what remains is not significant.

From §2.15 ANCOVA · the parallel-lines assumption belongs to §2.11 Interactions.

25Three scales, one model

Exam-level

A driving school fits a logistic regression predicting whether a learner passes the practical test on the first attempt, from the number of paid lessons taken beforehand.

Fitted model logit(pass) = −2.80 + 0.45 × (lessons)
  • Lessons in the sample ranged from 2 to 30
  • Useful values: e0.45 = 1.568, e2.25 = 9.488, e0.10 = 1.105, e−0.80 = 0.449
  1. Give the odds ratio for one extra lesson, and for five extra lessons.
  2. Compute the predicted pass probability at 6 lessons and at 8 lessons.
  3. Now do the same at 14 and 16 lessons. Compare the two increases.
  4. How many lessons put a learner at a coin-flip?
Worked solution
  1. Odds ratios OR (1 lesson) = eb₁ = e0.45 = 1.57 OR (5 lessons) = e5b₁ = e2.25 = 9.49 Each lesson multiplies the odds of passing by about 1.57. Five lessons multiply them by 1.575, which is 9.49, not by 5 × 1.57. Odds ratios compound.
  2. Probabilities at 6 and 8 lessons P = 1 / (1 + e−(b₀ + b₁x)) x = 6: η = −2.80 + 2.70 = −0.10 → P = 1/(1 + 1.105) = .475 x = 8: η = −2.80 + 3.60 = 0.80 → P = 1/(1 + 0.449) = .690 increase = .690 − .475 = .215
  3. Probabilities at 14 and 16 lessons x = 14: η = −2.80 + 6.30 = 3.50 → P = .971 x = 16: η = −2.80 + 7.20 = 4.40 → P = .988 increase = .988 − .971 = .017 The same two-lesson step carries the same odds ratio in both places, e0.90 = 2.46, and yet the probability moves 21.5 points in the middle of the curve and 1.7 points near the top. This is why a logistic model has no single "effect on probability". The odds ratio is constant everywhere; the probability change depends entirely on where you start.
  4. The 50% point P = .50 when the log-odds = 0, so b₀ + b₁x = 0 x = −b₀ / b₁ = 2.80 / 0.45 = 6.22 lessons Just over six lessons is where a learner becomes as likely to pass as to fail. The intercept alone answers a related question: at zero lessons the log-odds are −2.80, so P = .057, a 6% chance of passing untaught.

OR = 1.57 per lesson and 9.49 per five lessons. The same two extra lessons raise the pass probability by .215 around the middle of the curve and by .017 near the top, and the coin-flip point is 6.22 lessons.

From §2.16 Logistic Regression · the same curve reappears as a psychometric function in §3.16.

26The main effect that isn't the story

Exam-level

Sixty participants solve logic puzzles under one of four conditions in a 2 × 2 design: feedback arrives either immediately or after a delay, and the puzzles are either easy or hard. The abstract reports that immediate feedback improved accuracy by 10 percentage points.

Cell means (% correct), 15 participants per cell
EasyHard
Immediate feedback8268
Delayed feedback7852
From the ANOVA table MSerror = 90.0, dferror = 56
  1. Compute the marginal means and both main effects.
  2. Compute the simple effect of feedback at each difficulty level.
  3. Build the three F-ratios, given that SSA = 1500, SSB = 6000 and SSA×B = 540.
  4. Is the abstract's sentence defensible? Write a better one.
Worked solution
  1. Marginal means immediate = (82 + 68)/2 = 75 delayed = (78 + 52)/2 = 65 easy = (82 + 78)/2 = 80 hard = (68 + 52)/2 = 60 grand mean = (82 + 68 + 78 + 52)/4 = 70 Main effect of feedback = 75 − 65 = 10 points. Main effect of difficulty = 80 − 60 = 20 points.
  2. Simple effects feedback effect on easy puzzles: 82 − 78 = 4 points feedback effect on hard puzzles: 68 − 52 = 16 points interaction contrast = 4 − 16 = −12 points The 10-point main effect is the average of a 4 and a 16. It describes neither condition.
  3. The F-ratios F = MSeffect / MSerror, and each effect here has df = 1 feedback: F(1, 56) = 1500 / 90 = 16.67, p < .001 difficulty: F(1, 56) = 6000 / 90 = 66.67, p < .001 interaction: F(1, 56) = 540 / 90 = 6.00, p = .017 partial η² = SSeffect / (SSeffect + SSerror), with SSerror = 90 × 56 = 5040 = .23 (feedback), .54 (difficulty), .10 (interaction)
  4. Fixing the abstract The sentence is arithmetically true and practically misleading. A significant interaction means the feedback effect is not one number, so quoting the average as though it applied everywhere hides the finding. Something closer to the data: "Feedback timing interacted with difficulty, F(1, 56) = 6.00, p = .017, partial η² = .10. Immediate feedback helped very little on easy puzzles (4 points) and substantially on hard ones (16 points)." When an interaction is significant, describe the simple effects first and let the main effects sit in the table.

Both main effects and the interaction are significant, Fs(1, 56) = 16.67, 66.67 and 6.00. Because the feedback benefit is 4 points on easy puzzles and 16 on hard ones, the abstract's single 10-point claim describes no condition in the study.

From §2.4 Factorial ANOVA and §2.11 Interactions in Regression.

27Between, within, or neither

Warm-up

A researcher wants to compare three interface designs on task completion time. She has budget for about 90 testing sessions and asks you whether to run 90 people through one design each, or 30 people through all three.

What she has told you
  • Task completion time in seconds, three designs
  • Individual differences in typing speed are large
  • Each session takes about 20 minutes per design
  • The task is the same puzzle type in all three designs
  • Her pilot analysis reported ε = 0.72 from a Greenhouse–Geisser correction
  1. Which design would you recommend, and which test does it call for?
  2. What does the within-subjects version buy, and what does it cost?
  3. How many possible presentation orders are there, and how would you handle them?
  4. Her pilot mentioned sphericity. What is it, and what do you do with ε = 0.72?
  5. When would you drop repeated-measures ANOVA for a mixed model?
Worked solution
  1. The recommendation Within-subjects, analyzed with a one-way repeated-measures ANOVA. Typing speed varies a lot between people and has nothing to do with the interfaces, so in a between-subjects design it lands in the error term and buries the effect. Having each person serve as their own control removes it before the test ever runs.
  2. The trade What you gain is power: the same 90 sessions now compare designs within a person rather than across people, and the error term shrinks by whatever the individual differences were worth. What you pay for is order: someone on their third interface is faster because they have solved two puzzles already, and possibly more bored. Practice, fatigue and carryover are all real, and none of them exist in a between-subjects design.
  3. Counting the orders 3! = 3 × 2 × 1 = 6 possible orders With 30 participants you can run complete counterbalancing, five people per order, which balances practice effects across designs exactly. A Latin square using three sequences is the cheaper alternative when the number of conditions makes 6 impractical, and it balances first-order carryover rather than everything. Also vary the specific puzzles across designs so nobody sees the same problem twice.
  4. Sphericity and ε Sphericity requires the variances of the differences between every pair of conditions to be equal. When it fails, the F-test's degrees of freedom are too generous and the p-value comes out too small. Greenhouse–Geisser multiplies both df by ε to compensate. uncorrected: F(2, 58) corrected: F(2 × 0.72, 58 × 0.72) = F(1.44, 41.76) Report the fractional degrees of freedom and name the correction. An ε of 0.72 is a meaningful departure, so with three conditions this is not optional.
  5. When a mixed model wins Repeated-measures ANOVA needs a complete row for every participant, so one missing session throws that person out entirely. A mixed model keeps partial data, handles unequal numbers of trials per person, allows the covariance structure to be modeled rather than assumed, and extends to designs where the repeated factor is continuous. If sessions get canceled or the design grows a second random factor such as stimulus items, switch.

Run it within-subjects with a repeated-measures ANOVA, counterbalance across all 6 orders with 5 participants each, and report the Greenhouse–Geisser corrected test as F(1.44, 41.76).

From §1.5 Between vs Within Designs, §2.5 Repeated Measures ANOVA and §3.10 Mixed Models.

28Getting the inputs right

Exam-level

A master's student is planning a study comparing three teaching methods on an end-of-term test. Her draft methods section says: "Based on previous work we expect a medium effect. We will recruit 20 participants per group, which is standard in this literature."

What she has decided so far
  • Three independent groups, one test score per participant
  • Target power .80, α = .05, two-tailed
  • "Medium" effect, source unspecified
  • Planned n = 20 per group, 60 in total
  1. Which effect-size measure does this design need, and what is Cohen's medium value for it?
  2. Roughly what power does her planned sample actually have?
  3. What total sample would reach .80, and how would you check it?
  4. She adds that if the result is null she will "report observed power". Talk her out of it.
Worked solution
  1. The right effect size Three groups compared with a one-way ANOVA means Cohen's f, not d. Cohen's benchmarks for f are .10 small, .25 medium, .40 large. Writing "medium effect" without naming the metric is the first thing to fix, because d = 0.5 and f = 0.25 are different quantities that happen to share the label.
  2. Power at the planned n noncentrality λ = f²N = 0.25² × 60 = 3.75 df₁ = k − 1 = 2, df₂ = N − k = 57 power ≈ .37 If the effect she expects is really there, this design finds it about 37% of the time. She is more likely to end the year with an ambiguous null than with an answer, and a null from a study this size tells nobody anything.
  3. The sample that works λ = 0.25² × 159 = 9.94, df₁ = 2, df₂ = 156 → power = .805 159 in total, so 53 per group. Set the power calculator to one-way ANOVA, f = 0.25, k = 3, α = .05, power = .80 and it returns the same figure. Two things to say in the write-up: "medium" is the weakest possible justification for an effect size, and the smallest effect that would actually change teaching practice is the better anchor. If prior studies are the source, discount them, because published effects run inflated.
  4. Observed power Observed power is computed from the effect size she happens to obtain, which makes it a rearrangement of her own p-value rather than new information. A p just above .05 always yields observed power just under 50%, whatever the design. The question worth answering after a null result is different: what is the smallest effect this study could have detected with 80% power? Answer that from the design, before the data.
  5. One more thing to warn her about She mentioned a possible follow-up with a second factor. Interactions typically need considerably more data than main effects of the same nominal size, because the interaction contrast is a difference of differences and its standard error is correspondingly larger. Power the study for the effect she most wants to detect, and if that is the interaction, plan the sample around it.

She needs Cohen's f = 0.25, not d. At 20 per group her power is about .37; reaching .80 takes 159 participants, 53 per group. Observed power after the fact adds nothing.

From §3.2 Power Analysis · run the numbers in Power & Sample Size or sketch the whole design in Plan My Analysis.

29Significant model, significant nothing

Exam-level

An occupational psychologist regresses supervisor-rated job performance on three predictors in a sample of 150 employees. The model works. None of the predictors do, and one of them has the wrong sign.

Output
  • Overall model: R² = .31, F(3, 146) = 21.87, p < .001
  • Age: b = 0.09, p = .38
  • Years of experience: b = −0.18, p = .21
  • Years since qualifying: b = 0.24, p = .09
Each predictor regressed on the other two R² = .923 (age), .887 (experience), .861 (years since qualifying)
  1. Compute the tolerance and VIF for each predictor.
  2. Which cross the usual alarm thresholds?
  3. Explain the significant F with no significant coefficients, and the negative sign on experience.
  4. Give three possible fixes and say what each one costs.
  5. Does any of this damage the model's predictions?
Worked solution
  1. Tolerance and VIF tolerance = 1 − R²ⱼ VIF = 1 / tolerance
    PredictorR²ⱼToleranceVIF
    Age.923.07712.99
    Years of experience.887.1138.85
    Years since qualifying.861.1397.19
  2. Reading the thresholds The common alarm level is VIF > 5, with VIF > 10 as the lenient version. All three predictors clear the strict threshold and age clears the lenient one. A tolerance of .077 says that only 7.7% of the variance in age is unique to age; the other 92.3% is already carried by the other two predictors.
  3. Why the F and the t-tests disagree They are asking different questions. The F-test asks whether the three predictors together explain performance, and they clearly do: R² = .31. Each t-test asks whether one predictor adds anything once the other two are in the model, and since the three are nearly the same variable, none of them adds much on top of the others. This pattern is the textbook fingerprint of multicollinearity. The negative coefficient on experience is the same problem in a different costume: with predictors this entangled, the coefficient estimates become unstable, their standard errors inflate, and the fitted values can be reached by combinations that make no substantive sense. Do not interpret that minus sign as evidence that experience harms performance.
  4. Three fixes and their costs Drop predictors and keep one: simplest, and it changes the question. The surviving coefficient is now the effect of that variable plus everything it stands in for, so name it honestly (say "career stage", not "age"). Combine them into a composite or a principal component: keeps all the information and buys interpretability problems, since a component score is no longer a thing anyone can act on. Regularize with ridge regression: stabilizes the estimates by accepting a little bias, which is the right call when prediction matters more than individual coefficients. There is also a fourth option worth naming, which is to collect data where the predictors come apart, for instance including career changers whose age and experience diverge.
  5. Predictions are fine Multicollinearity inflates the variance of the coefficient estimates, not the fitted values. Within the range of predictor combinations actually observed, this model predicts as well as its R² suggests. What it cannot do is tell you which of the three variables is responsible, and that is usually the question that prompted the study.

VIFs of 12.99, 8.85 and 7.19, all above the usual alarm level of 5. The predictors are nearly the same variable, so the model as a whole is significant while no individual coefficient can be trusted, including its sign. Prediction survives; interpretation does not.

From §2.12 Multicollinearity · compare with the diagnostics in §2.8 and Problem 22 above.

30The twenty-four who did not come back

Stretch

A twelve-week weight-management trial randomized 180 adults to an intervention or a control arm. Twenty-four never returned for the week-12 weigh-in. The analyst dropped them and reported the result.

What the trial knows about the missing 24
  • 18 were in the intervention arm, 6 in the control arm (90 per arm at randomization)
  • Baseline weight: dropouts M = 104.2 kg, completers M = 96.1 kg
  • Dropouts reported more side effects at the week-4 check-in
  • All 24 have baseline and week-4 weights recorded; only week 12 is missing
  • Complete-case result: intervention −5.8 kg, control −2.1 kg, p = .003
  1. Compute the dropout rate in each arm. Is the difference itself worth reporting?
  2. Classify the missingness mechanism, and say what you cannot rule out.
  3. In which direction is the complete-case estimate likely to be wrong?
  4. Evaluate three quick fixes: mean imputation, last observation carried forward, and multiple imputation.
  5. What should the paper report?
Worked solution
  1. Dropout by arm intervention: 18 / 90 = 20.0% control: 6 / 90 = 6.7% overall: 24 / 180 = 13.3% Differential dropout is a finding, not a nuisance. Three times as many people left the intervention arm, and the week-4 side-effect reports suggest why. A trial where the treatment drives people out has learned something important about the treatment.
  2. Naming the mechanism Not MCAR: missingness is clearly related to arm, to baseline weight and to week-4 side effects, all of which are observed. That is the MAR pattern, and it is the workable case, because the observed variables carry the information needed to correct for it. What you cannot rule out is MNAR: if people quit precisely because the scale was not moving, missingness depends on the unobserved week-12 weight itself, and no amount of cleverness with the observed data settles that. The distinction is untestable from the data. It is an assumption you argue for and then probe.
  3. Direction of the bias Complete-case analysis keeps the people who stayed, and in the intervention arm those are disproportionately the ones for whom it was working and tolerable. The 20% who left were heavier at baseline and struggling with side effects. So the −5.8 kg is an estimate of what the treatment does for people who can stick with it, which flatters it relative to what it does for everyone offered it. The control arm loses far fewer people, so the comparison is unbalanced on top of being optimistic.
  4. Three fixes, graded Mean imputation is the worst of the three. It inserts values with zero variability, shrinks the standard deviation, weakens every correlation, and leaves the software believing it has a full 180 cases, so the standard errors and p-values come out overconfident. Last observation carried forward is better than it looks in this specific trial, since carrying a week-4 weight forward in a study where weight is falling understates the intervention's benefit and is therefore conservative. It is still an assumption dressed up as data, it does not propagate any uncertainty, and regulators have moved away from it. Multiple imputation is the right tool: build the imputation model from baseline weight, arm, week-4 weight and side-effect reports, generate several completed datasets, analyze each, and pool. The extra uncertainty from the holes then flows honestly into the standard errors. A mixed model fitted to all timepoints reaches the same place under MAR and needs no imputation step at all.
  5. What the paper should say How much was missing and from which arm; a comparison of dropouts and completers on every baseline variable available; the mechanism assumed and the reasoning behind it; the primary analysis under MAR, by multiple imputation or a mixed model; and a sensitivity analysis that asks what happens if the dropouts did badly, for instance assuming they returned to baseline weight. If the conclusion survives that, say so. If it does not, that is the result.

20% dropout in the intervention arm against 6.7% in the control arm, related to observed variables, so MAR at best and possibly MNAR. Complete-case analysis keeps the people it worked for and overstates the benefit. Use multiple imputation or a mixed model, and report a sensitivity analysis alongside it.

From §3.14 Missing Data · the reporting standard belongs to §3.4 From Output to Results.

🎲 Advanced · Modern methods

Seven problems from the Stats 3 syllabus: updating a belief with Bayes' rule, pulling sensitivity apart from bias, reading a survival curve that never reaches zero, and choosing between models that all fit.

31A very good test for a very rare disease

Exam-level

A screening program offers a blood test for a condition that affects 8 people in every 1,000. A patient tests positive and is told the test is "95% accurate". She would like to know what that means for her.

Test characteristics
  • Prevalence: 0.8% of the screened population
  • Sensitivity: 95% of people with the condition test positive
  • Specificity: 92% of people without it test negative
  1. Build the natural-frequency table for 100,000 screened people.
  2. What is the probability she has the condition, given the positive result?
  3. She takes a second, independent test and it is also positive. Now what?
  4. What is the probability that someone who tests negative is genuinely clear?
  5. Why does a 95% sensitivity feel so much more reassuring than the answer to question 2?
Worked solution
  1. Natural frequencies, 100,000 people
    Test positiveTest negativeTotal
    Has the condition76040800
    Does not have it7,93691,26499,200
    Total8,69691,304100,000
    800 × .95 = 760 true positives 99,200 × .08 = 7,936 false positives Counting people instead of manipulating probabilities is the single most reliable way to get this right, and it is how the answer becomes obvious rather than surprising.
  2. The answer she wants P(condition | positive) = 760 / 8,696 = .0874 About 9%. Of every 100 positive results the program issues, roughly 91 are false alarms. The test has not failed; it is doing exactly what its specificity promises. The trouble is that 8% of a very large healthy group outnumbers 95% of a very small sick one.
  3. A second positive posterior odds = prior odds × likelihood ratio LR⁺ = sensitivity / (1 − specificity) = .95 / .08 = 11.875 prior odds = .008 / .992 = .00806 after one positive: .00806 × 11.875 = .0958 → P = .0958/1.0958 = .0874 ✓ after two positives: .0958 × 11.875 = 1.1372 → P = 1.1372/2.1372 = .532 The first test moved her from 0.8% to 8.7%, and the second from 8.7% to 53%. Yesterday's posterior is today's prior, and the odds form makes the update a single multiplication. The word "independent" is carrying real weight here: if both tests share a systematic quirk, for instance both cross-reacting with the same protein, the second result adds far less than this calculation assumes.
  4. A negative result P(clear | negative) = 91,264 / 91,304 = .9996 99.96%. When the condition is rare, a negative result is extremely informative and a positive one barely is. Screening tests are usually built this way on purpose, because the positives get followed up and the negatives get sent home.
  5. Why the intuition fails Sensitivity answers "if you are sick, will the test say so?" She is asking the reverse: "the test said so, am I sick?" Those two conditional probabilities are only equal when the base rate is 50%, and here it is 0.8%. Confusing them is base-rate neglect, and it is not a beginner's mistake. Doctors asked this exact question in studies of medical decision-making get it wrong at high rates too, until the numbers are presented as counts of people.

A single positive leaves her at about 9%, two independent positives at 53%, and a negative result clears her with 99.96% probability. Nothing is wrong with the test; the base rate is doing the work.

From §3.7 Bayesian Thinking · the same 2 × 2 is the confusion matrix in §5.4 of the ML course.

32Updating what last quarter told you

Exam-level

A team redesigns the onboarding flow in an app. Last quarter's version had a completion rate they summarize as a Beta(8, 12) prior. The new flow goes out to 60 users and 33 of them finish onboarding.

Prior, data, and two software outputs
  • Prior: θ ~ Beta(8, 12)
  • Data: 33 completions out of 60 users
  • For a Beta(a, b): mean = a/(a+b), mode = (a−1)/(a+b−2), SD = √[ab / ((a+b)²(a+b+1))]
  • Software reports a 95% central credible interval of [.404, .621]
  1. Write down the posterior distribution.
  2. Compute its mean, mode and standard deviation.
  3. The observed completion rate is .55, but the posterior mean is lower. Why?
  4. Approximate P(θ > .50 | data), and write the credible interval as a sentence.
Worked solution
  1. The update Beta(a, b) prior + k successes in n trials → Beta(a + k, b + n − k) = Beta(8 + 33, 12 + 27) = Beta(41, 39) The Beta is conjugate to the binomial, which is why the update is addition rather than integration. A Beta(a, b) prior is worth a + b − 2 pretend observations, so this one carried the weight of 18 previous users.
  2. Summarizing the posterior mean = 41 / 80 = .5125 mode = 40 / 78 = .5128 SD = √[(41 × 39) / (80² × 81)] = √(1599 / 518400) = √.003085 = .0555 Mean and mode agree to three decimals because the posterior is nearly symmetric at these values. They part company when a or b is small.
  3. Why .5125 and not .55 The prior mean was 8/20 = .40, and the prior brought 18 pseudo-observations to a dataset of 60. The posterior lands between the two, pulled toward the prior in proportion to how much evidence each side carries. With a flat Beta(1, 1) prior instead, the posterior would be Beta(34, 28) with mean .548, essentially the raw .55. Which of those two answers is right depends on a question the arithmetic cannot settle: if last quarter's users really are informative about this quarter's, the shrinkage is information rather than distortion, and if they are not, that prior should not have been used.
  4. P(θ > .50) and the interval z = (.50 − .5125) / .0555 = −.225 P(θ > .50) ≈ 1 − Φ(−.225) = .589 The exact posterior probability is .5889, so the normal approximation is safe here. About a 59% chance the new flow beats a coin flip, which is a much more honest summary than "33 out of 60 is over half". The interval in words: given the prior and these 60 users, there is a 95% probability that the true completion rate lies between .404 and .621. That sentence is legal for a credible interval and illegal for a confidence interval, which is most of the reason people reach for Bayesian methods.
  5. What to tell the team The interval spans .404 to .621, so the data do not yet distinguish a mild improvement from a mild regression. 60 users is not enough. Rerun the numbers after a few hundred and the interval will tighten around whatever is true.

Posterior Beta(41, 39): mean .5125, mode .5128, SD .0555. The prior's 18 pseudo-observations pull the estimate below the observed .55, and P(θ > .50) is about .59.

From §3.8 Bayesian Estimation, building on §3.7.

33Who is the better witness?

Exam-level

In a recognition-memory experiment each participant studies 40 words, then sees those 40 mixed with 40 new ones and says "old" or "new" to each. Three participants produce very different response patterns.

Counts out of 40 old and 40 new words
ParticipantHitsFalse alarms
A3412
B408
C3820
z-values you will need z(.85) = 1.036, z(.30) = −0.524, z(.95) = 1.645, z(.50) = 0.000, z(.988) = 2.251, z(.207) = −0.816
  1. Compute d′ and c for participant A. Is A liberal or conservative?
  2. Compute β and the area under A's ROC curve.
  3. Do the same for participant C, then compare both participants' percent correct.
  4. Participant B recognized every old word. Why does the formula break, and what is the standard fix?
Worked solution
  1. Participant A H = 34/40 = .85 FA = 12/40 = .30 d′ = z(H) − z(FA) = 1.036 − (−0.524) = 1.56 c = −[z(H) + z(FA)] / 2 = −[1.036 − 0.524] / 2 = −0.26 A negative c is a liberal criterion: A says "old" more readily than a neutral observer would, which buys hits and pays for them in false alarms.
  2. Bias as a likelihood ratio, and the ROC area β = ec·d′ = e(−0.256)(1.561) = e−0.400 = 0.67 AUC = Φ(d′/√2) = Φ(1.561 / 1.414) = Φ(1.104) = .865 β below 1 says the same thing as negative c: at A's criterion the evidence is not yet more likely under "old" than under "new", and A commits anyway. The AUC of .865 is the bias-free summary. It would be identical for an observer with A's sensitivity and any criterion whatsoever.
  3. Participant C, and the percent-correct trap H = 38/40 = .95 FA = 20/40 = .50 d′ = 1.645 − 0.000 = 1.64 c = −[1.645 + 0]/2 = −0.82 A's percent correct = (34 + 28)/80 = .775 C's percent correct = (38 + 20)/80 = .725 C scores five points worse on percent correct and yet has slightly higher sensitivity, 1.64 against 1.56. The whole difference is strategy: C is far more liberal (c = −0.82) and pays for a near-perfect hit rate with 20 false alarms. Percent correct blends skill and strategy into one number and then hides which one moved. That is the argument for reporting d′ and c separately, and it is why a memory drug that "improves recognition" tells you nothing until you also know how often it produced false alarms.
  4. Participant B and the ceiling H = 40/40 = 1.00, and z(1) is infinite, so d′ cannot be computed. The standard repair is the log-linear correction: add 0.5 to every cell count and 1 to every trial total, applied to every participant rather than only to the offending one. H = (40 + 0.5)/(40 + 1) = 40.5/41 = .988 FA = (8 + 0.5)/41 = 8.5/41 = .207 d′ = 2.251 − (−0.816) = 3.07 c = −[2.251 − 0.816]/2 = −0.72 Applying the correction to everyone keeps the estimates comparable; correcting only the extreme participants would bias that person relative to the rest. Name whichever rule you used in the methods section, and treat a d′ above 3 with some caution, because rates this close to the ceiling make the estimate fragile.

A: d′ = 1.56, c = −0.26, β = 0.67, AUC = .865. C: d′ = 1.64 with a much more liberal c = −0.82, which is why C has the higher sensitivity and the lower percent correct. B needs the log-linear correction, giving d′ = 3.07.

From §3.17 Signal Detection Theory · the ROC half of it is §5.5 in ML & AI.

34Reading a psychometric function

Stretch

In a temporal bisection task, listeners learn a short anchor of 200 ms and a long anchor of 800 ms, then judge probe tones as "short" or "long". A logistic regression of the "long" response on probe duration is fitted separately for quiet and loud probes.

Fitted coefficients, probe duration in ms
Conditionb₀b₁ (per ms)
Quiet probe (baseline)−3.600.0090
Loud probe−2.940.0084
Useful forms PSE = −b₀ / b₁  ·  JND = ln 3 / b₁  ·  ln 3 = 1.0986
  1. Compute the PSE and JND in each condition.
  2. Compute the Weber fraction for each.
  3. Confirm the baseline JND by locating the 25% and 75% points directly.
  4. Is this a shift, a slope change, or both? What does it mean psychologically?
  5. The baseline PSE is not 500 ms. Should it be?
Worked solution
  1. PSE and JND quiet: PSE = 3.60 / 0.0090 = 400.0 ms quiet: JND = 1.0986 / 0.0090 = 122.1 ms loud: PSE = 2.94 / 0.0084 = 350.0 ms loud: JND = 1.0986 / 0.0084 = 130.8 ms The PSE is where the fitted curve crosses 50%, so it is the duration that feels exactly midway between the anchors. The JND is half the distance between the 25% and 75% points, which is the curve's steepness expressed in milliseconds.
  2. Weber fractions quiet: 122.1 / 400.0 = .305 loud: 130.8 / 350.0 = .374 Dividing precision by the point it was measured at makes the two conditions comparable despite sitting at different places on the duration axis.
  3. Checking the JND from the curve 75% point: b₀ + b₁x = ln 3 → x = (1.0986 + 3.60)/0.0090 = 522.1 ms 25% point: b₀ + b₁x = −ln 3 → x = (−1.0986 + 3.60)/0.0090 = 277.9 ms half the span = (522.1 − 277.9)/2 = 122.1 ms ✓ The ln 3 shortcut comes straight from the logistic: at 75% the odds are 3 to 1, so the linear predictor equals ln 3.
  4. Shift or slope Mostly a shift. The PSE drops by 50 ms while the JND rises by only 8.7 ms, so the curve slid left without flattening much. Louder probes are judged longer, which is a bias in perceived duration rather than a loss of timing precision. Had the JND doubled with the PSE unmoved, the finding would have been the opposite kind: the same subjective midpoint, measured with a noisier instrument. In a regression this dissociation is two terms. The condition main effect tests the shift, and the condition × duration interaction tests the slope change.
  5. 400 rather than 500 500 ms is the arithmetic midpoint of 200 and 800. 400 ms is their geometric mean, √(200 × 800), and bisection points landing near the geometric mean is one of the standard findings in timing research. It suggests the internal scale for duration is compressive, so the subjective middle sits below the arithmetic one. Getting 400 here is a signature, not an error.

Quiet: PSE 400.0 ms, JND 122.1 ms, Weber .305. Loud: PSE 350.0 ms, JND 130.8 ms, Weber .374. A 50 ms leftward shift with almost no change in slope, so louder tones feel longer without being harder to time.

From §3.16 Psychometric Functions · the fitting machinery is §2.16 Logistic Regression.

35Pick the family before you fit

Warm-up

A hospital analytics team has five questions and one statistician. Each outcome needs a GLM family and a link before anyone touches the data.

The five outcomes
  • a. Number of migraine attacks per patient per month, with many zeros
  • b. Whether a patient was readmitted within 30 days
  • c. Total cost of an admission in euros, strictly positive and heavily right-skewed
  • d. Number of items recalled out of a fixed list of 20 on a cognitive screen
  • e. Systolic blood pressure in mmHg, roughly symmetric
Output from the model fitted to (a) b₀ = 0.92, b₁ (drug vs placebo) = −0.51; residual deviance 412.6 on 178 df. Useful values: e0.92 = 2.509, e−0.51 = 0.601, e0.41 = 1.507.
  1. Give the family and link for each of the five outcomes.
  2. For each, say what exponentiating a coefficient gives you.
  3. Interpret b₁ from the migraine model, and give both group rates.
  4. Check that model for overdispersion, and say what to do about it.
Worked solution
  1. Families and links
    OutcomeFamily · linkeb means
    a. Attacks per monthPoisson · log (negative binomial if overdispersed)Rate ratio
    b. Readmitted yes/noBinomial · logitOdds ratio
    c. Cost in eurosGamma · logMultiplicative change in mean cost
    d. Recalled out of 20Binomial · logit, as successes out of 20Odds ratio per item
    e. Blood pressureGaussian · identity (ordinary regression)Nothing; read b directly
    Two are worth pausing on. Outcome (d) looks like a count but has a known ceiling of 20, which makes it a binomial proportion rather than a Poisson count; modeling the raw proportion with ordinary regression would happily predict recall of 23 items. Outcome (c) is the classic case for Gamma: costs cannot go negative, and their variance grows with their mean, which is exactly the mean-variance relationship Gamma encodes.
  2. Interpreting the drug coefficient rate ratio = eb₁ = e−0.51 = 0.60 placebo rate = eb₀ = e0.92 = 2.51 attacks per month drug rate = eb₀ + b₁ = e0.41 = 1.51 attacks per month check: 2.51 × 0.60 = 1.51 ✓ The drug multiplies the attack rate by 0.60, a reduction of about 40%. Coefficients are additive on the log scale and multiplicative on the count scale, which is the same pattern as odds ratios in logistic regression.
  3. Overdispersion deviance / df = 412.6 / 178 = 2.32 A Poisson model assumes variance equals mean, which puts this ratio near 1. At 2.32 the counts vary more than twice as much as Poisson allows, and the consequence is standard errors that are too small and p-values that are too impressive. Migraine counts cluster within people, so this is the expected outcome rather than a surprise. Refit as a negative binomial, which adds its own dispersion parameter, or use quasi-Poisson to rescale the standard errors. The point estimate of the rate ratio will barely move; the confidence interval around it will widen, correctly.

Poisson-log, binomial-logit, Gamma-log, binomial-logit again, and plain Gaussian. The drug cuts the attack rate from 2.51 to 1.51 per month, a rate ratio of 0.60, but a deviance-to-df ratio of 2.32 means the Poisson standard errors are not trustworthy until the model is refitted as negative binomial.

From §3.9 Generalized Linear Models · the logit branch is §2.16.

36The curve that never reaches zero

Exam-level

Ten patients enter a two-year follow-up study after treatment. The outcome is time to relapse in months. A plus sign marks a patient who was still relapse-free when the study or their participation ended.

Time to relapse, months (+ = censored) 3, 5+, 7, 9, 9, 12+, 14, 18+, 20, 24+
  1. Build the Kaplan–Meier table: event times, number at risk, events, and S(t).
  2. Estimate S(12) and S(15).
  3. What is the median survival time?
  4. Two colleagues suggest averaging the times instead. Compute both of their averages and say what is wrong with each.
  5. Why does the curve stop at a positive value?
Worked solution
  1. The Kaplan–Meier table S(t) = Π (1 − dⱼ / nⱼ) over all event times up to t
    tAt riskEvents1 − d/nS(t)
    3101.900.9000
    5+90censored.9000
    781.875.7875
    972.714.5625
    12+50censored.5625
    1441.750.4219
    18+30censored.4219
    2021.500.2109
    24+10censored.2109
    Two things drive the whole table. Censored patients count in the at-risk column right up to the moment they leave, and then simply drop out of it. The at-risk number falls by both events and censorings, which is why only 4 people are at risk at month 14 even though 6 have not yet relapsed on paper.
  2. Two readings off the curve S(12) = .5625 S(15) = .4219 Nothing happens at month 12 except a censoring, so the curve is flat across it and S(12) is still the value set at month 9. The staircase only steps down at event times.
  3. Median survival The median is the first time at which S(t) drops to .50 or below. At month 9 the curve is at .5625, still above half. At month 14 it reaches .4219. Median survival = 14 months. Read it off the vertical axis at 0.5 and drop to the horizontal one; do not average the observed times.
  4. The two bad averages all ten times: (3+5+7+9+9+12+14+18+20+24)/10 = 121/10 = 12.1 months events only: (3+7+9+9+14+20)/6 = 62/6 = 10.3 months The first treats every censored time as though the patient relapsed at that moment, which understates survival, since we only know they lasted at least that long. The second is worse: throwing out censored cases deletes precisely the people who were doing well, so it drops from 12.1 to 10.3 and moves further from the truth. The Kaplan–Meier median of 14 months is above both, because it is the only one of the three that uses the censored patients' partial information instead of destroying it.
  5. Why it stops at .2109 The longest observation, at 24 months, is censored. The curve has no information about what happens after that point, so it simply stops rather than pretending everyone eventually relapses. Estimates from the flat tail of a survival curve rest on very few people, which is why a curve that ends with 1 or 2 patients at risk should be read with the confidence band, not the line.

S steps to .9000, .7875, .5625, .4219 and .2109, giving a median survival of 14 months and S(12) = .5625, S(15) = .4219. Both naive averages (12.1 and 10.3 months) understate survival, and the curve ends at .2109 because the last patient was censored.

From §3.13 Survival Analysis.

37Four models, one dataset

Stretch

Working with the same 120 students from Problem 23, an analyst fits four nested models of exam performance and asks which one to report. The most complex model has the best log-likelihood, and she suspects that is not the end of the argument.

Model fits, n = 120
ModelPredictorskln L
M1study3−455.2
M2study + sleep4−449.8
M3study + sleep + motivation5−449.1
M4M3 plus all two-way interactions8−447.3
Formulas and constants AIC = −2 ln L + 2k  ·  BIC = −2 ln L + k ln n  ·  ln 120 = 4.787  ·  e−0.3 = 0.741, e−1.5 = 0.223, e−4.4 = 0.012, e4.4 = 81.5
  1. Compute AIC for all four models and rank them.
  2. Compute ΔAIC and interpret the spread with the usual rules of thumb.
  3. Compute the Akaike weights for M2 and M3, and the evidence ratio between them.
  4. M3 adds one parameter to M2. Run the likelihood-ratio test.
  5. M4 fits best. Why does it lose? And would BIC agree with AIC here?
Worked solution
  1. AIC for each model M1: −2(−455.2) + 2(3) = 910.4 + 6 = 916.4 M2: −2(−449.8) + 2(4) = 899.6 + 8 = 907.6 M3: −2(−449.1) + 2(5) = 898.2 + 10 = 908.2 M4: −2(−447.3) + 2(8) = 894.6 + 16 = 910.6 Ranking, best first: M2, M3, M4, M1. Lower is better, and the numbers themselves mean nothing in isolation. Only differences between models fitted to the same data are interpretable.
  2. ΔAIC ΔM2 = 0.0 ΔM3 = 0.6 ΔM4 = 3.0 ΔM1 = 8.8 The common reading: Δ (delta, the gap from the best model) under about 2 means the models are effectively indistinguishable, 4 to 7 means considerably less support, and above 10 means essentially none. So M2 and M3 are tied, M4 is trailing but not dead, and M1 is clearly out. The honest summary is that this dataset cannot separate M2 from M3.
  3. Akaike weights and the evidence ratio relative likelihood = e−Δ/2 M1: e−4.4 = 0.012 M2: e0 = 1.000 M3: e−0.3 = 0.741 M4: e−1.5 = 0.223 sum = 1.976 w(M2) = 1.000/1.976 = .506 w(M3) = 0.741/1.976 = .375 evidence ratio = .506/.375 = 1.35 = e0.3 M2 is 1.35 times better supported than M3. That is close to nothing. For comparison, M2 over M1 is e4.4 = 81 times better supported, which is what a real difference looks like.
  4. The nested test χ² = 2(ln L₃ − ln L₂) = 2(−449.1 + 449.8) = 2(0.7) = 1.4 df = 5 − 4 = 1, p = .24 Motivation does not earn its parameter. The likelihood-ratio test and AIC agree, which is what usually happens with one extra parameter, since AIC charges 2 and the χ² critical value at df = 1 is 3.84. They can disagree in either direction as models grow.
  5. Why the best fit loses, and what BIC says BIC: M1 924.8 M2 918.7 M3 922.1 M4 932.9 M4's log-likelihood is the highest because adding parameters can never make in-sample fit worse; setting a new coefficient to zero is always available, so the fit can only improve or stay put. AIC charges 2 per parameter for exactly this reason, and M4's four extra parameters bought only 2.5 units of log-likelihood, which is not enough. BIC picks M2 as well and does so more decisively, because its penalty of ln 120 = 4.79 per parameter is more than twice AIC's. Agreement between the two is reassuring; the interesting case is when they disagree, which usually means a moderately sized effect is on the edge of paying for itself. What would actually settle M2 against M3 is not another criterion but new data, or cross-validation on the data at hand.

AIC = 916.4, 907.6, 908.2 and 910.6, so M2 wins with M3 only ΔAIC = 0.6 behind, an evidence ratio of 1.35 that settles nothing. The likelihood-ratio test agrees that motivation is not worth its parameter (χ²(1) = 1.4, p = .24), and BIC picks M2 more decisively.

From §2.14 Model Comparison · the held-out version is §3.11 Cross-Validation.

🔍 Research Toolkit · Spot the problem

Six problems with almost no arithmetic in them. Each one hands you a study, a figure or a sentence that looks perfectly respectable, and asks what is wrong with it. These are the questions that separate someone who can run a test from someone you would trust with a dataset, and they draw on Methods, Data, Ethics and Writing.

38The module that raises exam scores

Exam-level

A university builds an optional online tutorial module for its introductory statistics course. Usage is tracked automatically. At the end of term the learning-technology team compares final exam marks and issues a press release.

What they report
  • Students who used the module: M = 71.0 (n = 104)
  • Students who did not: M = 62.0 (n = 96)
  • t(198) = 4.10, p < .001, d = 0.58
  • Headline: "New online module raises exam scores by nine points"
  1. The statistics are computed correctly. What is the problem?
  2. Name three specific variables that could produce this result with a module that does nothing.
  3. The team offers to rerun the analysis controlling for prior GPA. Does that fix it?
  4. Design the study that would actually answer the question.
  5. Rewrite the headline.
Worked solution
  1. Self-selection Nobody assigned anyone to anything. Students decided for themselves whether to use the module, so the two groups differ in whatever made them decide, and the module is only one candidate explanation for the nine-point gap. The t-test is valid as a description of two groups that already existed. It is the word "raises" that has no support, because a comparison of self-selected groups cannot distinguish an effect of the treatment from a difference between the people who chose it.
  2. Three plausible confounds Prior mathematical preparation: students who already find statistics tractable are more likely to engage with extra material and more likely to score well. Conscientiousness: the trait that produces module use also produces lecture attendance, problem sets and revision. Available time: students working long hours in paid jobs have less of everything, including both module use and study time. Each of these causes both the exposure and the outcome, which is the definition of a confound.
  3. Does controlling for GPA fix it? It helps and it does not fix it. Adjusting for prior GPA removes the part of the gap explained by measured prior ability, which is real progress, and the adjusted estimate will almost certainly be smaller than nine points. What remains is everything that predicts both usage and outcome but was never measured: motivation this term, hours available, whether a friend recommended the module. Statistical adjustment can only remove confounding you have data on. This is where the language of causal diagrams earns its keep, because drawing the assumed structure makes visible which variables you would need and which you actually have.
  4. The study that would settle it Randomize access. Give the module to a random half of the cohort and withhold it from the other half for the term, or stagger the rollout so every student eventually gets it but the order is random, which is usually easier to get past an ethics committee and a students' union. Pre-register the primary outcome and the analysis. Analyze by assigned group rather than by actual usage, because the moment you analyze by who chose to log in, self-selection is back. If randomization is genuinely impossible, a regression-discontinuity or difference-in-differences design around a rollout date beats a naive comparison.
  5. An honest headline "Students who chose to use the new module scored nine points higher, though they may have differed before the term began." Longer, duller, and true. The internal version should add that a randomized rollout next year would answer the question properly.

Self-selection. The groups were formed by student choice, so the nine-point gap confounds the module with prior ability, conscientiousness and available time. Controlling for GPA shrinks the estimate without licensing causal language; only random assignment does that.

From §1.4 Experiments & Random Assignment and §1.6 Quasi-Experiments.

39Four thousand responses and no answer

Warm-up

A city council is deciding whether to pedestrianize the main square. It posts a survey link on the council website and its social media accounts, leaves it open for six weeks, and reports the result to the planning committee.

The report to committee
  • 4,300 responses received
  • 78% in favor of pedestrianization
  • "Margin of error ±1.5%"
  • Conclusion: "Over three quarters of residents support the scheme"
  • City population: about 260,000 adults
  1. Who could actually end up in this sample, and who could not?
  2. Does 4,300 responses make the estimate trustworthy?
  3. What is wrong with quoting a margin of error here?
  4. Name one more threat specific to open online surveys.
  5. What should the council have done, and is the survey worthless?
Worked solution
  1. The frame and the volunteers Two filters run before anyone answers. Coverage: only people who visit the council website or follow its social accounts ever see the link, which excludes residents without reliable internet, those who do not follow local government, and anyone whose news comes through other channels. Self-selection: of the people who saw it, the ones who answered are those who cared enough to click, and strong feelings on both sides drive response far more than mild ones. Organized campaigns on either side can mobilize supporters through a single post. The realized sample is people who saw the link and chose to respond, which is not the adult population of the city.
  2. Why 4,300 does not help Sample size fixes variance, not bias. A convenience sample's estimate is centered in the wrong place, and collecting more of the same kind of response tightens the cloud around that wrong center. 4,300 self-selected responses give a more precise wrong answer than 430 would. The only cure for bias is a better sampling method.
  3. The margin of error The ±1.5% figure is the most misleading number in the report. That formula assumes a probability sample in which every resident had a known, non-zero chance of selection, and here nobody was selected at all; people arrived. Quoting it attaches the authority of survey statistics to a number that has none of its properties. It describes sampling error alone, and sampling error is not this survey's problem.
  4. One more threat Nothing stops one person responding repeatedly, or a campaign asking supporters to. An open link with no authentication has no way to enforce one response per resident, and no way to confirm that respondents live in the city at all.
  5. What to do, and what the survey is worth For an estimate: draw a probability sample from the electoral register, stratify by neighborhood and age band so every part of the city is represented, follow up non-responders more than once, and report the response rate alongside the result. That design supports both the percentage and its margin of error. The open survey is not worthless, though. As a consultation it surfaces arguments, local knowledge and objections the council would not otherwise hear. It should be reported as what it is: 4,300 people chose to tell us this, with no claim about the other 255,700.

A self-selected sample drawn from an incomplete frame. The 78% describes people who saw the link and chose to respond, the sample size makes that no more trustworthy, and the ±1.5% margin of error is meaningless because the formula assumes probability sampling that never happened.

From §1.8 Sampling Methods and §1.9 Designing Surveys.

40Four questionable practices in one paragraph

Exam-level

You are reviewing a manuscript for a journal. The methods and results read as follows. Nothing in it is fabricated, and every number is real. It should still not be published in this form.

The submitted excerpt

"We recruited 60 participants. After inspecting the data we excluded 7 participants whose scores fell more than 2 SD from the mean, as their responses were clearly unrepresentative. Because the effect was not significant at that point, we collected a further 30 participants and re-ran the analysis, which then reached significance. Participants completed measures of mood, arousal, sustained attention and reaction time. As predicted, the drug improved reaction time, F(1, 81) = 4.62, p = .034, η² = .05."

  1. Identify four distinct questionable research practices in this paragraph.
  2. For each one, state the fix.
  3. Are the degrees of freedom consistent with the recruitment described?
  4. None of this is fraud. Why does it matter more than fraud does?
Worked solution
  1. Post-hoc exclusion "After inspecting the data we excluded 7 participants" is an outlier rule chosen after seeing what it would do, and a 2 SD cutoff is lenient enough to remove roughly 5% of a normal distribution by construction. Calling the excluded responses "clearly unrepresentative" is a judgment made with knowledge of the result. Fix: specify the exclusion rule before data collection, apply it blind to condition, and report the analysis both with and without exclusions.
  2. Optional stopping "Because the effect was not significant, we collected a further 30 participants" is the textbook definition of data-dependent stopping. Every additional look is another chance for the p-value to wander below .05, and this practice alone can push a nominal 5% false-positive rate toward 20% or more. Fix: fix the sample size in advance from a power analysis, or use a pre-planned sequential design with corrected thresholds, which is a legitimate and sometimes required approach. What is not legitimate is deciding after each look.
  3. Selective reporting of outcomes Four measures were collected and one is reported. The reader has no way to know whether reaction time was the planned primary outcome or the one that survived four attempts. Fix: report every measure collected, nominate a primary outcome in advance, and correct for multiplicity across the rest or label them exploratory.
  4. HARKing "As predicted" cannot be assessed by anyone, and given the three practices above there is good reason to doubt it. Presenting a finding located by exploration as a prediction converts a hypothesis-generating result into apparent confirmation. Fix: separate confirmatory from exploratory sections explicitly, and let a preregistration carry the claim about what was predicted.
  5. The degrees of freedom 60 recruited − 7 excluded + 30 added = 83 participants two groups, so dferror = 83 − 2 = 81 ✓ The arithmetic is fine. Every number in the paragraph is internally consistent, which is exactly why this manuscript would pass a statistical-consistency screen and still be untrustworthy. The problem is the path taken to those numbers, and no recomputation can detect it.
  6. Why this outranks fraud Fabrication is rare, with under 1% of surveyed researchers admitting it, and it is unambiguous when caught. These practices are common: in a survey of more than 2,000 psychologists, around 63% admitted failing to report all dependent measures and about 56% admitted deciding whether to collect more data after checking significance. They also feel like reasonable judgment calls at the time, which is why they persist. Rare deliberate fraud damages a few papers; widespread gray-zone flexibility fills the literature with effects that evaporate on replication, and that is the mechanism behind the replication crisis.

Post-hoc outlier exclusion, optional stopping, selective reporting of one outcome out of four, and HARKing. The degrees of freedom check out perfectly, which is the point: consistent numbers are no defense against a flexible analysis path.

From §4.5 Questionable Research Practices, §1.11 The Replication Crisis and §1.12 Preregistration.

41The cleaning log that broke the data

Exam-level

A research assistant hands you a cleaned dataset for a two-wave wellbeing study and, to their considerable credit, an actual log of what they did to it. Reading the log is faster than discovering each problem later.

The cleaning log, verbatim
  • a. "Deleted 14 rows that had a blank cell anywhere."
  • b. "Replaced remaining blanks in the wellbeing items with 0 so the scale total would compute."
  • c. "Three participants entered 999 for age. Left them in rather than lose cases. Mean age is 33.2 (n = 240)."
  • d. "Reverse-scored items 3, 7 and 11 by multiplying by −1." (all items are 1–7 Likert)
  • e. "Removed 6 participants with the lowest wellbeing scores, who looked like non-serious responders."
  • f. "Merged wave 1 and wave 2 on participant initials."
  • g. "Saved the cleaned file over the original."
  1. Which steps are defensible and which are errors?
  2. Give the correct version of steps (b) and (d).
  3. Using step (c), work out the true mean age.
  4. Which single step is the most serious, and why?
Worked solution
  1. Step-by-step verdict
    StepVerdictWhy
    a. Listwise deletionDefensible, needs justificationFine if the data are plausibly MCAR and the loss is small, but it must be reported and it contradicts step (b)
    b. Blanks → 0Error0 is not a valid response on a 1–7 scale
    c. 999 left inError999 is a missing-value code, not an age
    d. Reverse by ×(−1)ErrorProduces −1 to −7, destroying the scale
    e. Dropped lowest scorersSerious errorSelection on the outcome variable
    f. Merged on initialsErrorCollides, breaks on name changes, and is identifying
    g. Overwrote the raw fileSerious errorThe original data no longer exists
  2. Fixing (b) and (d) For (b), leave missing values missing and decide an item-level rule instead: compute the scale when at least 80% of its items were answered, prorating from the items that were, and set it to missing otherwise. Filling with 0 invents the most extreme response available and drags every affected participant's total toward the floor. correct reverse-scoring: x' = (min + max) − x = 8 − x so 1 → 7, 2 → 6, 4 → 4, 7 → 1 Multiplying by −1 gives a range of −1 to −7, which is not the reversed scale, and it corrupts every scale total that includes those three items without producing a single value that looks obviously wrong. Recompute the reversed items from the raw file, which is a problem given step (g).
  3. The true mean age total of all 240 ages = 240 × 33.2 = 7,968 the three sentinels contribute 3 × 999 = 2,997 remaining total = 7,968 − 2,997 = 4,971 true mean = 4,971 / 237 = 21.0 years Three cells inflated the reported mean by 12.2 years in a sample of 240. This is what missing-value sentinels do when a pipeline treats them as data, and it is why the first step of any cleaning workflow is to check every variable's minimum and maximum against its plausible range.
  4. The most serious step Step (e), with step (g) close behind. Removing the six lowest wellbeing scores is not cleaning; it is deleting cases on the basis of the outcome variable, which changes the mean by construction and does so in a direction the researcher can see while doing it. If non-serious responding is a genuine concern, screen for it with something independent of the outcome: attention-check items, implausibly fast completion times, straight-lining down a column. Step (g) is a different kind of damage. Raw data should be read-only and every transformation should live in a script, so that the cleaned file can always be regenerated and any mistake in this log can be undone. As things stand, none of these errors can be reversed.

Five clear errors: 0 for missing on a 1–7 scale, a 999 sentinel treated as an age (which inflated the mean from 21.0 to 33.2), reverse-scoring by −1 instead of 8 − x, dropping the lowest scorers, and merging on initials. Overwriting the raw file is what makes all of them permanent.

From §2.4 The Cleaning Workflow, §2.5 Outliers and §2.9 Reproducible Workflows · the worked version is the survey-cleaning guide.

42The figure that made 0.27 look enormous

Exam-level

A retail consultancy sends its client a one-page summary of a store-layout trial. The figure is the whole page, and you have been asked whether the client should spend two million on the rollout.

Figure 2, as submitted
  • Bar chart, two bars, drawn with 3-D perspective
  • y-axis runs from 7.2 to 7.8; satisfaction is measured on a 1–10 scale
  • Layout A bar at 7.35, Layout B bar at 7.62
  • No error bars anywhere on the figure
  • A red arrow between the bars, labeled "+27% improvement"
  • Caption: "Figure 2. Layout B clearly outperforms Layout A."
From the appendix Layout A: M = 7.35, SD = 1.42, n = 41 · Layout B: M = 7.62, SD = 1.38, n = 38
  1. List everything wrong with the figure.
  2. Where does "+27%" come from, and what is the honest number?
  3. Using the appendix, is the difference distinguishable from zero?
  4. Describe the figure you would send instead.
Worked solution
  1. The catalog of problems The truncated axis is the big one: starting at 7.2 instead of 0 turns a 0.27-point gap into a bar nearly three times the height of its neighbor, since A occupies 25% of the plot area and B occupies 70%. Bars encode value by length, so a bar chart with a truncated axis is misreporting its own data. Beyond that: no uncertainty is shown, so a reader cannot tell a solid difference from noise; the 3-D perspective distorts the apparent heights and adds nothing; two bars summarize 79 people and hide every distribution; and the caption asserts a conclusion ("clearly outperforms") that the figure does not establish and, as question 3 shows, is wrong.
  2. Where the 27% came from 7.62 − 7.35 = 0.27 points relative change = 0.27 / 7.35 = 3.7% Someone read the difference of 0.27 as "27%". The actual relative increase in the mean is 3.7%, and even that is a shaky thing to report, because a 1–10 satisfaction scale has no meaningful zero, so percentage changes on it depend on an arbitrary origin. The honest statement is the difference itself: Layout B scored 0.27 points higher on a 10-point scale.
  3. Is it distinguishable from zero? pooled = [40(1.42²) + 37(1.38²)] / 77 = (80.66 + 70.46)/77 = 1.963 spooled = 1.401 SE = 1.401 × √(1/41 + 1/38) = 1.401 × .2252 = 0.3155 t(77) = 0.27 / 0.3155 = 0.86, p = .39 d = 0.27 / 1.401 = 0.19 No. A difference this size arises about 39% of the time when the two layouts are identical. The figure's entire visual argument rests on a difference the data cannot distinguish from nothing, and the two-million rollout has no evidence behind it. With d = 0.19, detecting an effect this small with 80% power would need over 400 stores per layout rather than 40.
  4. The figure to send instead Plot the individual store scores as two columns of points, jittered so they do not overlap, with the group mean marked and a 95% confidence interval drawn around it. Run the y-axis across the full 1 to 10 scale, or across a clearly labeled subrange with the truncation stated in the caption. Drop the third dimension. Make the caption descriptive: "Figure 2. Mean customer satisfaction by store layout, with 95% confidence intervals. Individual stores shown as points; n = 41 and 38." Then let the overlapping intervals say what they say. A figure whose job is to persuade is not a figure, and if the recommendation is to keep testing, the figure should make that obvious at a glance.

Truncated axis, no error bars, gratuitous 3-D, a conclusion in the caption, and a "+27%" that is really a 0.27-point difference (3.7% in relative terms). The difference is not distinguishable from zero, t(77) = 0.86, p = .39, d = 0.19, so the rollout rests on nothing.

From §3.3 Tables & Figures That Don't Lie · check the size of the effect in the effect-size converter.

43Three things wrong with one sentence

Warm-up

A results paragraph comes back from a co-author. The analysis is sound and every number was computed correctly, but the sentence breaks exactly three APA 7 formatting rules. Find them, then write the version that goes in the paper.

The study Sixty participants, 30 per group, completed a puzzle task under immediate or delayed feedback. Accuracy and total practice time were recorded for all 60. The sentence as written

"Participants receiving immediate feedback (M = 78.4, SD = 8.6) were more accurate than those receiving delayed feedback (M = 74.1, SD = 8.9), t(58) = 1.90, p = 0.062, d = .49. Accuracy correlated with practice time, r(60) = .34, p = .008."

  1. Find the three formatting errors and name the rule behind each.
  2. Write the corrected sentence.
  3. Confirm that the numbers themselves are consistent.
  4. Your supervisor circles something else in the sentence that is not a formatting error. What?
Worked solution
  1. Error 1: a leading zero on p written: p = 0.062 correct: p = .062 Drop the leading zero from any statistic that cannot exceed 1 in magnitude: p, r, R², η², β and Cramér's V. A probability of 0.062 is never going to be 1.062, so the zero carries no information.
  2. Error 2: a missing zero on d written: d = .49 correct: d = 0.49 The mirror image of the first rule, and the one people miss because they over-apply the first. Cohen's d can comfortably exceed 1, so it keeps its leading zero. The same goes for t, F, M and SD. The test is not "is it a decimal" but "can this quantity exceed 1".
  3. Error 3: the wrong df on the correlation written: r(60) = .34 correct: r(58) = .34 The number in parentheses after r is degrees of freedom, not sample size, and for a correlation that is n − 2. With 60 participants the value is 58. Writing r(60) tells a careful reader you had 62 people, which contradicts the methods section. The t-test in the same sentence has it right, and the two matching by coincidence here is a good reminder to check rather than pattern-match.
  4. The corrected sentence "Participants receiving immediate feedback (M = 78.4, SD = 8.6) scored higher on accuracy than those receiving delayed feedback (M = 74.1, SD = 8.9), though the difference was not statistically significant, t(58) = 1.90, p = .062, d = 0.49. Accuracy correlated with practice time, r(58) = .34, p = .008."
  5. Checking the numbers SE = √[(8.6² + 8.9²)/30] = √(153.17/30) = 2.26 t = (78.4 − 74.1)/2.26 = 1.90 ✓, two-tailed p = .062 ✓ spooled = √[(8.6² + 8.9²)/2] = 8.75, so d = 4.3/8.75 = 0.49 ✓ for r = .34 with df = 58: t = .34√(58/(1 − .34²)) = 2.75, p = .008 ✓ Everything reconciles, which is what the consistency check in the APA formatter does automatically. Formatting errors and arithmetic errors are separate species, and a sentence can have either without the other.
  6. What the supervisor circled "Were more accurate" states a difference at p = .062. That is not a formatting error; it is a claim the test does not support. The correction above replaces it with a description of the observed means plus an explicit statement that the difference was not significant, which is the honest form. Resist both temptations here: do not write "approached significance", and do not pretend the 4.3-point gap was not observed. Report the estimate, report that it is compatible with no difference at this sample size, and let the confidence interval carry the uncertainty.

Leading zero on p (should be .062), missing leading zero on d (should be 0.49), and r(60) where the degrees of freedom are n − 2 = 58. The numbers themselves all reconcile; the fourth problem, calling p = .062 a difference, is a writing error rather than a formatting one.

From §3.2 Reporting Statistics in APA Style and §3.5 Non-Significant Results · the rules are on the APA cheat sheet and applied by the APA formatter.

Worked through these? Take a scored mock exam, drill definitions with the flashcards, or move from pencil-and-paper numbers to real files with the practice datasets. Teachers: using this site in a course covers handouts, embeds and a fourteen-week map.