Practice Problems

1. Classify each column

   Column	Level	Why
   Membership number	Nominal	Numeric, but it is a name for a person, not a quantity
   Age in years	Ratio	Equal intervals and a real zero, so 40 is twice 20
   Postcode district	Nominal	A label with digits in it
   Satisfaction 1–5	Ordinal	Ordered, but the gap 1→2 need not equal 4→5
   Books borrowed	Ratio	A count; zero books means zero borrowing
   Temperature °C	Interval	Equal intervals, arbitrary zero: 20 °C is not twice as warm as 10 °C
   Branch	Nominal	Unordered categories
   Queue position	Ordinal	Ranks; the gap between 1st and 2nd is not a fixed quantity

2. Which can be averaged Age, books borrowed and temperature. Interval and ratio data support addition and division, which is all a mean is.
3. The postcode mean The digits in M14 encode a place, not an amount. Averaging them produces a number that refers to nothing: there is no location "17.4", and moving from M14 to M20 is not "six units" of anything.
4. Mean satisfaction of 3.8 Strictly, a mean assumes equal spacing, and Likert points are only ordered. In practice psychology and market research report Likert means constantly, and with five or more points and roughly symmetric responses the distortion is small. Report the median and the full response distribution alongside it, and the reader can judge for themselves.

Three of the eight columns are nominal, two are ranks, one is interval and two are ratio. Only age, books borrowed and temperature are safely averageable, and satisfaction is the one worth a footnote.

From §1.2 Types of Data.

1. Mean x̄ = Σx / n Σx = 12+14+15+16+17+19+21+25+41 = 180 x̄ = 180 / 9 = 20.0 loaves
2. Median The values are already in order and n = 9 is odd, so the median is the 5th value. Median = 17 loaves
3. Deviations and their squares (x − x̄): −8, −6, −5, −4, −3, −1, 1, 5, 21 (x − x̄)²: 64, 36, 25, 16, 9, 1, 1, 25, 441 Σ(x − x̄)² = 618
4. Variance, then standard deviation s² = Σ(x − x̄)² / (n − 1) = 618 / 8 = 77.25 s = √77.25 = 8.789 ≈ 8.79 loaves Divide by n − 1, not n: these nine days are a sample of the bakery's trading, not the whole of it.
5. Which center to plan with The mean of 20 is dragged up by the single festival day. Drop that day and the mean of the remaining eight is 139 / 8 = 17.4, which lands right next to the median. For an ordinary Tuesday the median of 17 is the honest planning figure, and the festival day belongs in the plan as its own event.

Mean = 20.0, median = 17, s = 8.79 loaves. Order for about 17, and order extra for festival days separately.

Rounding: keep the full 8.7892 if the figure feeds another calculation, and round to 8.79 only when reporting.

From §1.3 Describing Data · check it in the Descriptives calculator.

1. Formula z = (x − μ) / σ
2. Statistics z = (68 − 60) / 10 = 8 / 10 = 0.80
3. Research Methods z = (74 − 65) / 12 = 9 / 12 = 0.75
4. Compare The raw marks say Methods (74 > 68). The z-scores say Statistics, by a whisker: 0.80 SDs above her cohort against 0.75. The bigger raw mark came on the paper where everyone scored higher and the spread was wider.
5. T-scores T = 50 + 10z Statistics: 50 + 10(0.80) = 58.0 Methods: 50 + 10(0.75) = 57.5

Statistics z = 0.80 (T = 58.0), Methods z = 0.75 (T = 57.5). Her Statistics paper was marginally the stronger performance, and the raw marks pointed the other way.

From §1.5 Z-Scores & The Normal Distribution.

1. Below 60 bpm: standardize z = (60 − 72) / 8 = −12 / 8 = −1.50 P(Z < −1.50) = .0668 About 6.7% of adults, or roughly 1 in 15.
2. Between 64 and 84 bpm: two z-scores z₁ = (64 − 72) / 8 = −1.00 z₂ = (84 − 72) / 8 = +1.50
3. Subtract the two cumulative areas P(−1.00 < Z < 1.50) = P(Z < 1.50) − P(Z < −1.00) = .9332 − .1587 = .7745 About 77.5%.
4. The top 5%: go backwards from the area The cutoff leaves .95 below it, so z = 1.645 x = μ + zσ = 72 + 1.645(8) = 72 + 13.16 = 85.16 Round to 85 bpm for the protocol.

6.7% below 60 bpm, 77.5% between 64 and 84 bpm, and the top 5% sit above roughly 85 bpm.

Rounding: 1.645 is the conventional two-decimal z for the 95th percentile (1.6449 exactly). Using 1.64 gives 85.12 and using 1.65 gives 85.20, so at whole-bpm precision the choice does not matter here. It would matter if the protocol were written to one decimal.

From §1.5 Z-Scores & The Normal Distribution · check the areas in Statistical Tables.

1. Marginal probability P(cycles) = 70 / 200 = .35
2. Conditional probability: the denominator shrinks P(cycles | under 30) = 48 / 100 = .48 Conditioning on "under 30" throws away the other row, so the denominator is that row's total, not 200.
3. Joint probability P(under 30 and cycles) = 48 / 200 = .24
4. Independence check If independent: P(A and B) = P(A) × P(B) P(under 30) × P(cycles) = .50 × .35 = .175 Observed joint = .24 ≠ .175
5. Read it the other way too P(cycles | under 30) = .48 against P(cycles) = .35. Knowing someone's age band changes the probability, which is exactly what dependence means.

.35, .48, .24, and no: the two variables are dependent. Under-30s cycle at nearly twice the rate of over-30s (48% against 22%).

From §1.6 Probability Basics · test the dependence formally with a chi-square test (Problem 19 below).

1. Formula SE = σ / √n
2. n = 25 SE = 60 / √25 = 60 / 5 = 12 ms
3. n = 100 SE = 60 / √100 = 60 / 10 = 6 ms Four times the sample halves the standard error, because √4 = 2. Precision is bought with the square root, which is why the second hundred participants buy far less than the first hundred.
4. Solve for n 4 = 60 / √n → √n = 60 / 4 = 15 n = 15² = 225 drivers
5. SD against SE The SD describes how much individual drivers differ from each other, and it does not shrink when you collect more of them. The SE describes how much a sample mean would bounce around from study to study, and it shrinks as √n grows.

SE = 12 ms at n = 25, 6 ms at n = 100, and 225 drivers are needed for an SE of 4 ms.

From §1.7 Sampling Distributions.

1. Why normality applies here The central limit theorem is about the distribution of the sample mean, not about the raw deliveries. With n = 36 the sampling distribution of x̄ is close to normal even though individual delivery times are not. The skew has to be extreme, or n small, for this to fail.
2. Standard error SE = σ / √n = 9 / √36 = 9 / 6 = 1.5 min
3. P(x̄ > 30) — standardize the mean, not a delivery z = (30 − 28) / 1.5 = 2 / 1.5 = 1.333 P(Z > 1.333) = 1 − .9088 = .0912
4. P(27 < x̄ < 29) z₁ = (27 − 28) / 1.5 = −0.667 z₂ = (29 − 28) / 1.5 = +0.667 P(−0.667 < Z < 0.667) = .7475 − .2525 = .4950
5. Sanity check A single delivery taking over 30 minutes is common. A mean of 36 deliveries doing so is not, at about 9%. Averaging is what kills the variability.

The CLT covers the sample mean regardless of the skew. P(x̄ > 30) = .091, and P(27 < x̄ < 29) = .495, so almost exactly half of such samples land within a minute of the true mean.

Rounding: carry z = 1.333, not 1.3. Rounding to 1.3 gives .0968 instead of .0912, an error of more than half a percentage point.

From §1.8 Central Limit Theorem.

1. Formula: t, not z, because σ is unknown CI = x̄ ± t* × (s / √n), df = n − 1
2. Standard error SE = 1.5 / √25 = 1.5 / 5 = 0.30 h
3. Critical value df = 25 − 1 = 24, t*(.025, 24) = 2.064
4. Margin of error 2.064 × 0.30 = 0.619 h
5. The interval 6.8 ± 0.619 = [6.18, 7.42] hours
6. Against the 8-hour guidance 8 sits well outside the interval, so 8 hours is not a plausible value for the population mean given this sample. Equivalently, a one-sample t-test against 8 would reject at α = .05. What the interval does not say is that 95% of first-years sleep between 6.18 and 7.42 hours: the interval is about the mean, not about individuals.
7. With n = 100 SE = 1.5 / 10 = 0.15, df = 99, t* = 1.984 margin = 1.984 × 0.15 = 0.298 CI = [6.50, 7.10] hours Four times the sample gives an interval about half as wide, and t* also drifts down towards 1.96 as df grows.

95% CI [6.18, 7.42] hours. First-years sleep less than the recommended 8 hours, and quadrupling the sample narrows the interval to [6.50, 7.10].

From §1.9 Confidence Intervals · look up t* in Statistical Tables.

1. What p actually is If the feed truly made no difference, 3% of trials run this way would produce a difference of at least 0.42 kg in one direction or the other. It is the probability of data this extreme given the null hypothesis, written P(data | H₀).
2. Marek's error He has flipped the conditional into P(H₀ | data). Those two are not interchangeable, and turning one into the other needs a prior probability that the test never supplies.
3. The neighbor's version Same inversion, wearing a hat. 1 − p is not the probability that the alternative is true, and it is not the probability the result replicates either.
4. The decision p = .030 < α = .05 → reject H₀ If the feed genuinely does nothing, this rejection is a Type I error. Setting α = .05 caps that error rate at 5% over the long run; it says nothing about whether this particular result is the mistake.
5. The 8-plot repeat No. A non-significant result means the data are compatible with no effect, not that the effect is zero. With 8 plots the test has very little power, so a real effect of this size would be missed most of the time. Failing to detect it is a Type II error. Report the confidence interval: if it stretches from −0.3 to +1.1 kg, the study is simply too small to settle the question.
6. Twenty null trials Expected false positives = 20 × .05 = 1 One society in twenty would find "significant" evidence for a feed that does nothing. If only the exciting result gets into the newsletter, the newsletter reports an effect that is not there.

p is P(data at least this extreme | H₀ true), so both Marek and his neighbor have inverted the conditional. The decision is to reject at α = .05, with Type I error as the risk. The 8-plot follow-up shows nothing either way, and one false positive per twenty null trials is the expected background rate.

From §1.10 Hypothesis Testing Logic and §1.13 Effect Size & Power.

1. Hypotheses H₀: μ = 65 H₁: μ ≠ 65 Two-tailed, because the consumer group would report the course being better than claimed as well as worse.
2. Standard error SE = s / √n = 6.4 / √16 = 6.4 / 4 = 1.60
3. Test statistic t = (x̄ − μ₀) / SE = (61.5 − 65) / 1.60 t = −3.5 / 1.60 = −2.1875 df = n − 1 = 15
4. Critical value t*(.025, 15) = 2.131 |−2.1875| = 2.188 > 2.131 → reject H₀
5. p-value p = 2 × P(T₁₅ > 2.1875) = .045
6. Effect size d = (x̄ − μ₀) / s = −3.5 / 6.4 = −0.55 A medium shortfall of just over half a standard deviation.
7. Write-up Graduates typed more slowly than the advertised 65 wpm, M = 61.5, SD = 6.4, t(15) = −2.19, p = .045, d = −0.55, 95% CI [58.09, 64.91].

Reject H₀. The 65 wpm claim is not supported, though at p = .045 with n = 16 the evidence is thin, and the confidence interval reaches almost up to 65.

Rounding matters here. Carry t = −2.1875 rather than rounding the standard error first; this test clears its critical value by 0.06, so an early round-off can flip the decision. Report t to two decimals and p to three.

From §1.11 One-Sample & Paired t-Tests · format the sentence with the APA formatter.

1. Why paired Each after-score is tied to a specific person's before-score. Subtracting within a person removes that person's baseline memory ability from the comparison, which is where most of the noise lives. An independent test would treat the twenty scores as twenty unrelated people and throw that advantage away.
2. Mean difference Σd = 3+5−1+4+6+2+0+7+3+1 = 30 d̄ = 30 / 10 = 3.00 items
3. SD of the differences (d − d̄): 0, 2, −4, 1, 3, −1, −3, 4, 0, −2 (d − d̄)²: 0, 4, 16, 1, 9, 1, 9, 16, 0, 4 → Σ = 60 s_d = √(60 / 9) = √6.667 = 2.582
4. Standard error and t SE = s_d / √n = 2.582 / √10 = 2.582 / 3.162 = 0.8165 t = d̄ / SE = 3.00 / 0.8165 = 3.674 df = n − 1 = 9
5. Decision t*(.025, 9) = 2.262 3.674 > 2.262 → reject H₀ p = 2 × P(T₉ > 3.674) = .005
6. Effect size for a paired design d_z = d̄ / s_d = 3.00 / 2.582 = 1.16 Large. Note that d_z is scaled by the SD of the differences, so it is not comparable with a between-groups d computed from raw score SDs. Say which one you used.

Recall improved by 3.00 items on average, t(9) = 3.67, p = .005, d_z = 1.16. With ten students and no control group, this shows the scores went up; it does not rule out practice or test-order effects.

From §1.11 One-Sample & Paired t-Tests · effect-size conversions in the converter.

1. Pooled variance s²_p = [(n₁−1)s₁² + (n₂−1)s₂²] / (n₁ + n₂ − 2) = [11(64) + 11(81)] / 22 = (704 + 891) / 22 = 1595 / 22 = 72.50 → s_p = √72.50 = 8.515
2. Standard error of the difference SE = s_p × √(1/n₁ + 1/n₂) = 8.515 × √(1/12 + 1/12) = 8.515 × √0.1667 = 8.515 × 0.4082 = 3.476
3. Test statistic t = (78.0 − 71.5) / 3.476 = 6.5 / 3.476 = 1.870 df = 12 + 12 − 2 = 22
4. Decision t*(.025, 22) = 2.074 1.870 < 2.074 → fail to reject H₀, p = .075
5. Effect size d = (M₁ − M₂) / s_p = 6.5 / 8.515 = 0.76 By Cohen's benchmarks that is medium-to-large, just short of the 0.80 mark, sitting inside a non-significant test.
6. Assess the conclusion "Equally well" is not supported. A non-significant result means the data do not rule out zero difference, and here the best estimate of the difference is 6.5 marks with d = 0.76. With 12 per group the study had about 43% power to detect an effect that size, so it was more likely to miss it than to find it. The honest sentence is that the trial was too small to decide.
7. Sample size for a replication Two-sample t, d = 0.76, α = .05, power = .80 n = 29 per group (58 children total) 28 per group falls just short at 79.8% power, which is why the answer rounds up rather than down. Planning for a medium d = 0.50 instead would need 64 per group. Powering a study on the effect size your own underpowered pilot produced is optimistic, because a pilot that reached significance has almost certainly overestimated the effect.

t(22) = 1.87, p = .075, d = 0.76. Not significant, not "no difference": a large point estimate measured with far too little precision. A replication needs at least 29 per group.

From §1.12 Independent Samples t-Test and §1.13 Effect Size & Power · redo the sample size in the Power calculator.

1. Group means and the grand mean Silence: Σ = 20, M = 4 Music: Σ = 30, M = 6 Café: Σ = 40, M = 8 Grand: Σ = 90, N = 15, GM = 6
2. Between-groups sum of squares SS_B = Σ n_j (M_j − GM)² = 5(4−6)² + 5(6−6)² + 5(8−6)² = 5(4) + 5(0) + 5(4) = 40
3. Within-groups sum of squares Silence: (−1)²+(1)²+0²+(−2)²+(2)² = 10 Music: (−2)²+(1)²+0²+(−1)²+(2)² = 10 Café: (1)²+(−2)²+0²+(2)²+(−1)² = 10 SS_W = 10 + 10 + 10 = 30 The three groups happen to have identical spread, which is the homogeneity of variance an ANOVA wants.
4. The table

   Source	SS	df	MS	F
   Between (condition)	40	2	20.00	8.00
   Within (error)	30	12	2.50
   Total	70	14

   df_B = k − 1 = 2, df_W = N − k = 12, and SS_B + SS_W = SS_Total confirms the arithmetic.
5. Decision F(2, 12) = 8.00, F*(.05, 2, 12) = 3.89 8.00 > 3.89 → reject H₀, p = .006
6. Effect size η² = SS_B / SS_Total = 40 / 70 = .571 57% of the variance in errors missed is associated with the sound condition. η² is biased upward in small samples; ω² = .483 is the less flattering estimate.
7. Why not three t-tests Three pairwise tests at α = .05 push the chance of at least one false positive to about 14% rather than 5% (1 − .95³ = .143 if the three were independent; sharing the same data pulls it a little below that, but nowhere near back to .05). The ANOVA asks one question at one α, and the follow-up comparisons then get their own correction (Problem 14).

Sound condition affected proofreading, F(2, 12) = 8.00, p = .006, η² = .57. The omnibus test says the three means are not all equal; it does not say which pairs differ.

From §2.1 One-Way ANOVA.

1. Pairwise standard error: use MS_W, not just the two groups SE = √(MS_W × (1/n₁ + 1/n₂)) = √(2.50 × (1/5 + 1/5)) = √(2.50 × 0.40) = √1.00 = 1.00 Pooling all three groups' error gives df = 12 rather than the 8 a standalone pair would have.
2. The three comparisons

   Comparison	Difference	t	p
   Silence vs music	2	2.00	.069
   Silence vs café	4	4.00	.002
   Music vs café	2	2.00	.069

   Each t = difference / 1.00, evaluated on df = 12
3. Bonferroni α_adjusted = .05 / 3 = .0167 Silence vs café: .002 < .0167 → significant The other two: .069 > .0167 → not significant
4. Tukey's HSD as a cross-check HSD = q*(.05, k = 3, df = 12) × √(MS_W / n) = 3.77 × √(2.50 / 5) = 3.77 × 0.707 = 2.67 Any difference above 2.67 is significant. The 4-point silence-café gap clears it; the two 2-point gaps do not. Both methods reach the same verdict, with Tukey slightly less conservative, as it is built for exactly this set of all-pairs comparisons.
5. What the correction cost Uncorrected, silence vs music at p = .069 would still not have reached .05, so nothing was lost here. In general Bonferroni trades power for protection, and with many comparisons that trade gets expensive. Tukey and Holm both keep the family-wise error rate at .05 while giving some of that power back.

Only silence against café chatter survives correction (difference = 4 errors, t(12) = 4.00, p = .002). Music sits between the two and is not distinguishable from either at α = .05.

From §2.2 Post-Hoc Tests.

1. Degrees of freedom df_caffeine = 2 − 1 = 1 df_sleep = 2 − 1 = 1 df_interaction = 1 × 1 = 1 df_error = N − (number of cells) = 40 − 4 = 36
2. Error mean square MS_error = 36,000 / 36 = 1,000
3. The table

   Source	SS	df	MS	F	p	partial η²
   Caffeine	16,000	1	16,000	16.00	<.001	.308
   Sleep	49,000	1	49,000	49.00	<.001	.576
   Caffeine × Sleep	9,000	1	9,000	9.00	.005	.200
   Error	36,000	36	1,000

   partial η² = SS_effect / (SS_effect + SS_error)
4. Decisions F*(.05, 1, 36) = 4.11 All three Fs exceed 4.11, so both main effects and the interaction are significant.
5. Simple effects of caffeine Rested: 420 − 410 = 10 ms Deprived: 520 − 450 = 70 ms
6. Interpret the interaction Caffeine does something quite different depending on sleep. It buys a rested person 10 ms, which is close to nothing, and a sleep-deprived person 70 ms. Reporting the caffeine main effect on its own ("caffeine improved reaction time by 40 ms") would describe an average that applies to neither group well. With a significant interaction, lead with the simple effects and treat the main effects as background.

Both main effects and the interaction are significant. Sleep deprivation is the larger influence (partial η² = .58), and caffeine helps mainly when you are short of sleep: 70 ms recovered when deprived against 10 ms when rested.

From §2.3 Factorial ANOVA (Two-Way).

1. Degrees of freedom df_conditions = k − 1 = 2 df_error = (n − 1)(k − 1) = 11 × 2 = 22 The error term is smaller than a between-subjects design's would be, because each typist serves as their own control.
2. Mean squares and F MS_conditions = 84.5 / 2 = 42.25 MS_error = 159.2 / 22 = 7.236 F = 42.25 / 7.236 = 5.84 F*(.05, 2, 22) = 3.44 → significant, p = .009
3. What Mauchly's test found Sphericity means the variances of the differences between every pair of conditions are equal. Mauchly's p = .03 rejects that assumption. When sphericity fails, the F ratio is positively biased: the real Type I error rate runs above the nominal 5%, so the uncorrected p of .009 is too generous.
4. Greenhouse–Geisser correction Multiply both df by ε = .71 df_conditions = 2 × .71 = 1.42 df_error = 22 × .71 = 15.62 F itself is unchanged at 5.84; only the reference distribution moves, and it moves against you.
5. Re-decide F(1.42, 15.62) = 5.84, p = .020 Still below .05, so the conclusion survives. The p-value roughly doubled, which is what the correction costs. An uncorrected p near .04 would have flipped.
6. Effect size partial η² = 84.5 / (84.5 + 159.2) = 84.5 / 243.7 = .347

Layout affected typing speed, F(1.42, 15.62) = 5.84, p = .020, partial η² = .35, with Greenhouse–Geisser applied because Mauchly's test rejected sphericity. Report the corrected df with decimals; they are meant to look like that.

From §2.4 Repeated Measures ANOVA.

1. The failed assumption s²_B / s²_A = 47.61 / 9.61 = 4.95 Clinic B's variance is roughly five times Clinic A's. Levene's p = .004 says a variance gap this large would be rare if the population variances were equal, so homogeneity of variance is rejected. Note that Levene's test is itself sensitive to sample size; with unequal n and a visible five-fold gap, the plot and the ratio agree with it here.
2. Normality of what Of the sampling distribution of each group mean, not of the raw session counts. With n = 45 the central limit theorem largely covers Clinic B despite the skew. Clinic A's n = 15 is the more fragile half, and it is the one with the tidier distribution.
3. Welch's t — no pooling SE = √(s₁²/n₁ + s₂²/n₂) s₁²/n₁ = 9.61 / 15 = 0.6407 s₂²/n₂ = 47.61 / 45 = 1.0580 SE = √1.6987 = 1.3033 t = (8.2 − 6.4) / 1.3033 = 1.8 / 1.3033 = 1.381
4. Welch–Satterthwaite degrees of freedom df = (s₁²/n₁ + s₂²/n₂)² / [ (s₁²/n₁)²/(n₁−1) + (s₂²/n₂)²/(n₂−1) ] = (1.6987)² / [ 0.6407²/14 + 1.0580²/44 ] = 2.8855 / [ 0.029318 + 0.025442 ] = 2.8855 / 0.054760 df = 52.69 t*(.025, 52.69) = 2.006 → 1.381 < 2.006, p = .173
5. What pooling would have done The pooled SD is 6.20, dominated by the larger, wider Clinic B. Student's t would give 0.97 on df = 58, p = .334. Both tests fail to reject, but they disagree substantially about how far apart the clinics are, which is why the choice has to be made before looking at which answer you prefer.
6. Welch or Mann–Whitney For Mann–Whitney: session counts are bounded below at zero and badly skewed, and the median is arguably the more meaningful summary for a service report. Against: Mann–Whitney tests stochastic dominance rather than means, and with unequal spreads its interpretation is no cleaner than Welch's. Welch handles unequal variances directly and keeps the outcome on its natural scale. Running one, reporting it, and saying why is better practice than running both and reporting the smaller p.

Homogeneity of variance fails. Welch's t(52.69) = 1.38, p = .173: the clinics do not differ detectably in sessions attended, and Welch is the defensible default whenever the variances are unequal.

From §2.5 Assumptions & When They Break · the assumption-checks poster lists the fix for each violation.

1. Pool and rank, smallest first

   Value	9	11	12	14	15	17	18	20	22	31
   Shelter	B	B	A	B	A	B	A	B	A	A
   Rank	1	2	3	4	5	6	7	8	9	10

   No ties, so no shared ranks to average.
2. Rank sums R_A = 3 + 5 + 7 + 9 + 10 = 34 R_B = 1 + 2 + 4 + 6 + 8 = 21 Check: 34 + 21 = 55 = N(N+1)/2 = 10(11)/2 ✓
3. Convert rank sums to U U = R − n(n+1)/2 U_A = 34 − 5(6)/2 = 34 − 15 = 19 U_B = 21 − 15 = 6 Check: U_A + U_B = 25 = n_A × n_B ✓
4. Decision U = min(19, 6) = 6 U*(.05, n₁ = n₂ = 5, two-tailed) = 2 6 > 2 → fail to reject H₀ Mann–Whitney is one of the few tests where you reject when the statistic is small. The exact two-tailed p here is .222.
5. The floor on p Five against five gives C(10,5) = 252 possible arrangements. Exactly one of them puts every Shelter B dog ahead of every Shelter A dog, and one puts the reverse, so the smallest attainable two-tailed p is 2/252 = .0079. The rejection region at α = .05 is U ≤ 2, which is four arrangements in each tail out of 126, so a design this small can only detect something close to complete separation.

U = 6, p = .222: no detectable difference in days to adoption. Shelter B looks faster in every summary you could compute, and with five dogs each the test cannot tell that apart from chance.

From §2.6 Non-Parametric Alternatives.

1. Expected counts E = (row total × column total) / N First year: 120 × 60 / 180 = 40 (all three columns) Final year: 60 × 60 / 180 = 20 (all three columns) All expected counts are 20 or above, comfortably clear of the "at least 5" rule, so no correction or exact test is needed.
2. Cell contributions χ² = Σ (O − E)² / E (30−40)²/40 = 100/40 = 2.500 (45−40)²/40 = 25/40 = 0.625 (twice) (30−20)²/20 = 100/20 = 5.000 (15−20)²/20 = 25/20 = 1.250 (twice)
3. Total and df χ² = 2.500 + 0.625 + 0.625 + 5.000 + 1.250 + 1.250 = 11.25 df = (rows − 1)(columns − 1) = (2 − 1)(3 − 1) = 2
4. Decision χ²*(.05, 2) = 5.99 11.25 > 5.99 → reject H₀, p = .004
5. Effect size Cramér's V = √( χ² / (N × min(r−1, c−1)) ) = √( 11.25 / (180 × 1) ) = √0.0625 = .25 A small-to-moderate association.
6. Where the association lives The single largest contribution is the final-year "alone" cell at 5.00, where 30 students chose solitary revision against 20 expected. Only a quarter of first-years revise alone, and three-quarters work with someone; among finalists it is half and half. Reading the biggest cell contributions is how you turn one significant χ² into a sentence about people.

Revision preference depends on year group, χ²(2, N = 180) = 11.25, p = .004, Cramér's V = .25. Final-years revise alone far more than first-years do.

From §2.7 Chi-Square Tests · check the critical value in Statistical Tables.

1. Means x̄ = 42 / 6 = 7.0 ȳ = 30 / 6 = 5.0
2. Deviations and cross-products

   x	y	x−x̄	y−ȳ	(x−x̄)²	(y−ȳ)²	product
   4	3	−3	−2	9	4	6
   5	5	−2	0	4	0	0
   6	4	−1	−1	1	1	1
   8	5	1	0	1	0	0
   9	7	2	2	4	4	4
   10	6	3	1	9	1	3
   Sums				28	10	14

3. Pearson's r r = Σ(x−x̄)(y−ȳ) / √( Σ(x−x̄)² × Σ(y−ȳ)² ) = 14 / √(28 × 10) = 14 / √280 = 14 / 16.733 r = .837
4. Significance test t = r√(n−2) / √(1 − r²), df = n − 2 = 4 = .837 × √4 / √(1 − .70) = 1.673 / √0.30 = 1.673 / 0.548 = 3.055 t*(.025, 4) = 2.776 → 3.055 > 2.776, reject H₀, p = .038 Significant, but only just, and on six people. A correlation of .84 needs to be enormous to clear the bar at n = 6.
5. r² r² = .837² = .70 70% of the variance in mood is shared with sleep in this sample. "Shared", not "caused by", and in a sample this small the estimate is very unstable.
6. Two non-causal readings Reverse direction: low mood disrupts sleep, which is well documented, so mood could be driving sleep rather than the other way round. Third variable: workload, pain, a new baby, or alcohol would each push sleep down and mood down at the same time, producing the correlation with no arrow between the two measured variables at all. Only random assignment to a sleep condition would separate these.

r(4) = .84, p = .038, r² = .70. Strong and significant in this sample, and entirely consistent with mood driving sleep or with something else driving both.

From §2.8 Correlation · try the correlation calculator.

1. Slope b = Σ(x−x̄)(y−ȳ) / Σ(x−x̄)² = 14 / 28 = 0.50
2. Intercept: the line goes through (x̄, ȳ) a = ȳ − b·x̄ = 5.0 − 0.50(7.0) = 5.0 − 3.5 = 1.50 ŷ = 1.50 + 0.50x
3. Interpret the slope Each extra hour of sleep is associated with half a point more mood on the 1–10 scale. The intercept of 1.50 is the predicted mood at zero hours of sleep, which is outside the data and not meaningful on its own.
4. Prediction and residual at x = 6 ŷ = 1.50 + 0.50(6) = 4.50 observed y = 4 residual = y − ŷ = 4 − 4.50 = −0.50 This person's mood was half a point below what their sleep predicted.
5. All six residuals, as a check −0.5, +1.0, −0.5, −0.5, +1.0, −0.5 Σ residuals = 0 ✓ (always true for a least-squares fit with an intercept) SS_residual = 0.25 + 1 + 0.25 + 0.25 + 1 + 0.25 = 3.00
6. Tie it back to r² SS_total = 10, SS_residual = 3.00, SS_regression = 7.00 r² = 7.00 / 10 = .70 ✓ matches Problem 20 s_e = √(SS_residual / (n − 2)) = √(3/4) = 0.87
7. Why not predict at 2 hours The data run from 4 to 10 hours. Predicting at 2 assumes the straight line keeps going where nobody was measured, and there is no evidence in these six people that it does. Mood is bounded at 1, and the true relationship almost certainly bends at the extremes. Extrapolation is the most common way a correct regression produces a wrong number.

ŷ = 1.50 + 0.50x. At 6 hours the model predicts 4.50, the person scored 4, so the residual is −0.50. The fit explains 70% of the variance in mood over the observed range of 4 to 10 hours, and says nothing about 2.

From §2.9 Simple Linear Regression · fit it in the correlation & regression calculator.

1. The fan Heteroscedasticity: residual variance grows with the fitted value, which is typical for prices. The coefficients stay unbiased, so the slope estimates are still fine on average. What breaks is everything built on the standard errors: the t-tests, the p-values and the confidence intervals are all wrong, usually too narrow at the top end.
2. Leverage Cutoff = 2(k + 1)/n = 2(2 + 1)/40 = 6/40 = 0.15 h = 0.31, about twice the cutoff Case 31 sits far out on the predictors: a very large house, or an odd area-to-bedrooms combination. High leverage alone is not a crime, it only means the case has the opportunity to pull the line.
3. Cook's D Common cutoffs: D > 1, or the stricter 4/n = 4/40 = 0.10 D = 0.68 → below 1, but nearly seven times 4/n High leverage plus a large Cook's D means this case is not merely unusual, it is actively moving the fitted coefficients. Refit without it and see how much the slopes shift.
4. Multicollinearity VIF = 1 / (1 − r²) = 1 / (1 − .92²) = 1 / (1 − .8464) = 1 / .1536 = 6.51 Above the usual warning level of 5. Floor area and bedroom count carry nearly the same information, so the model cannot tell their contributions apart. The symptom to expect is a significant overall F with two unstable, possibly non-significant individual coefficients, one of which may even carry the wrong sign.
5. What to do, in order Investigate case 31 as a data point before deleting anything, since a genuine mansion is information rather than error, and report the fit with and without it if it matters. For the fan, model log(price) instead of price, or keep the model and use heteroscedasticity-robust standard errors. For the VIF, drop one predictor or combine the two into a single size measure, and be explicit that the model now answers a narrower question. Report the diagnostics you ran, including the ones that came back clean.

Three problems, three different fixes: heteroscedasticity invalidates the standard errors, case 31 has both the leverage and the influence to bend the line, and a VIF of 6.51 means the two predictors are nearly the same variable. None of them is a reason to abandon the model, and all three change what the coefficients mean.

From §2.10 Regression Diagnostics and §3.8 Assumptions of Regression.

1. Reading a raw coefficient Among students who sleep the same amount, each extra hour of weekly study is associated with 1.30 more exam points. The phrase doing the work is "who sleep the same amount": every coefficient in a multiple regression is the effect of its predictor with the others held fixed. Nothing here licenses the word causes, since nobody was assigned their study hours.
2. Standardizing β = b × (SD of the predictor) / (SD of the outcome) β_study = 1.30 × 4.0 / 12.4 = 5.20 / 12.4 = .419 β_sleep = 3.20 × 1.2 / 12.4 = 3.84 / 12.4 = .310 The journalist has it backwards. Study hours vary far more between students (SD = 4.0 hours) than sleep does (SD = 1.2 hours), so an hour of study is a much smaller step through the actual data than an hour of sleep. Once both are expressed in standard deviations, studying is the stronger predictor. Raw coefficients are only comparable when the predictors share a scale, and hours-per-week and hours-per-night do not.
3. A prediction ŷ = 29.57 + 1.30(10) + 3.20(7) = 29.57 + 13.00 + 22.40 = 64.97 ≈ 65.0 points Both values sit near the middle of their observed ranges, which is the only region where a prediction is safe.
4. Adjusted R² and the intercept adj R² = 1 − (1 − R²)(n − 1)/(n − k − 1) = 1 − (.66)(119/117) = 1 − .671 = .329 ≈ .33 Barely below R², because two predictors is a cheap model at n = 120. The intercept of 29.57 is the predicted score for a student who studies zero hours a week and sleeps zero hours a night. No such student exists in the data, and the model has no information about that corner of the space, so the number is an artifact of where the plane happens to cross the axis.

β_study = .42 against β_sleep = .31, so studying is the stronger predictor and the headline is wrong. Predicted score at 10 study hours and 7 sleep hours is 65.0, adjusted R² = .33.

From §3.1 Multiple Regression · the same distinction shows up in the effect-size converter.

1. The covariate's grand mean X̄ = (52.0 + 45.0) / 2 = 48.5 Equal group sizes, so the simple average is the grand mean. ANCOVA compares the groups at this pretest value.
2. Adjusted means adjusted Ȳ_j = Ȳ_j − b(X̄_j − X̄) program: 61.4 − 0.62(52.0 − 48.5) = 61.4 − 2.17 = 59.23 comparison: 55.0 − 0.62(45.0 − 48.5) = 55.0 + 2.17 = 57.17 The program group is slid down because it started 3.5 points above average, and the comparison group is slid up for the mirror-image reason.
3. The adjusted effect 59.23 − 57.17 = 2.06 points shortcut: adjusted = raw − b × (covariate gap) = 6.4 − 0.62(7.0) = 6.4 − 4.34 = 2.06 Of the 6.4-point advantage, 4.34 points were already there in September. What the term added, at least as far as this analysis can see, is 2.06 points, and F(1, 57) = 1.48, p = .229 says even that is within what chance produces at this sample size.
4. Rewriting the claim "Children in the program finished 6.4 points ahead, but they also began the year 7.0 points ahead. Adjusting for the starting gap leaves a 2.06-point difference that is not statistically distinguishable from zero, F(1, 57) = 1.48, p = .229." Note also that families chose the program, so even the adjusted figure is an observational comparison. Pretest score is the one baseline difference that was measured; motivation, home support and reading habits were not.
5. Homogeneity of regression slopes ANCOVA fits one slope to both groups and reports a single adjusted gap. That gap only exists if the pretest predicts the posttest equally in each group. Test it by adding the group × pretest interaction: if the interaction is significant, the lines are not parallel and there is no single number to report. You would then fit the interaction model and describe how the program's effect changes across the pretest range, which is often the more interesting finding anyway.

Adjusted means 59.23 and 57.17, an adjusted difference of 2.06 points against a raw difference of 6.4. Two thirds of the headline gap was baseline imbalance, and what remains is not significant.

From §3.4 ANCOVA · the parallel-lines assumption belongs to §3.5 Interactions.

1. Odds ratios OR (1 lesson) = e^b₁ = e^0.45 = 1.57 OR (5 lessons) = e^5b₁ = e^2.25 = 9.49 Each lesson multiplies the odds of passing by about 1.57. Five lessons multiply them by 1.57⁵, which is 9.49, not by 5 × 1.57. Odds ratios compound.
2. Probabilities at 6 and 8 lessons P = 1 / (1 + e^{−(b₀ + b₁x)}) x = 6: η = −2.80 + 2.70 = −0.10 → P = 1/(1 + 1.105) = .475 x = 8: η = −2.80 + 3.60 = 0.80 → P = 1/(1 + 0.449) = .690 increase = .690 − .475 = .215
3. Probabilities at 14 and 16 lessons x = 14: η = −2.80 + 6.30 = 3.50 → P = .971 x = 16: η = −2.80 + 7.20 = 4.40 → P = .988 increase = .988 − .971 = .017 The same two-lesson step carries the same odds ratio in both places, e^0.90 = 2.46, and yet the probability moves 21.5 points in the middle of the curve and 1.7 points near the top. This is why a logistic model has no single "effect on probability". The odds ratio is constant everywhere; the probability change depends entirely on where you start.
4. The 50% point P = .50 when the log-odds = 0, so b₀ + b₁x = 0 x = −b₀ / b₁ = 2.80 / 0.45 = 6.22 lessons Just over six lessons is where a learner becomes as likely to pass as to fail. The intercept alone answers a related question: at zero lessons the log-odds are −2.80, so P = .057, a 6% chance of passing untaught.

OR = 1.57 per lesson and 9.49 per five lessons. The same two extra lessons raise the pass probability by .215 around the middle of the curve and by .017 near the top, and the coin-flip point is 6.22 lessons.

From §3.7 Logistic Regression · the same curve reappears as a psychometric function in §4.11.

1. Marginal means immediate = (82 + 68)/2 = 75 delayed = (78 + 52)/2 = 65 easy = (82 + 78)/2 = 80 hard = (68 + 52)/2 = 60 grand mean = (82 + 68 + 78 + 52)/4 = 70 Main effect of feedback = 75 − 65 = 10 points. Main effect of difficulty = 80 − 60 = 20 points.
2. Simple effects feedback effect on easy puzzles: 82 − 78 = 4 points feedback effect on hard puzzles: 68 − 52 = 16 points interaction contrast = 4 − 16 = −12 points The 10-point main effect is the average of a 4 and a 16. It describes neither condition.
3. The F-ratios F = MS_effect / MS_error, and each effect here has df = 1 feedback: F(1, 56) = 1500 / 90 = 16.67, p < .001 difficulty: F(1, 56) = 6000 / 90 = 66.67, p < .001 interaction: F(1, 56) = 540 / 90 = 6.00, p = .017 partial η² = SS_effect / (SS_effect + SS_error), with SS_error = 90 × 56 = 5040 = .23 (feedback), .54 (difficulty), .10 (interaction)
4. Fixing the abstract The sentence is arithmetically true and practically misleading. A significant interaction means the feedback effect is not one number, so quoting the average as though it applied everywhere hides the finding. Something closer to the data: "Feedback timing interacted with difficulty, F(1, 56) = 6.00, p = .017, partial η² = .10. Immediate feedback helped very little on easy puzzles (4 points) and substantially on hard ones (16 points)." When an interaction is significant, describe the simple effects first and let the main effects sit in the table.

Both main effects and the interaction are significant, Fs(1, 56) = 16.67, 66.67 and 6.00. Because the feedback benefit is 4 points on easy puzzles and 16 on hard ones, the abstract's single 10-point claim describes no condition in the study.

From §2.3 Factorial ANOVA and §3.5 Interactions in Regression.

1. The recommendation Within-subjects, analyzed with a one-way repeated-measures ANOVA. Typing speed varies a lot between people and has nothing to do with the interfaces, so in a between-subjects design it lands in the error term and buries the effect. Having each person serve as their own control removes it before the test ever runs.
2. The trade What you gain is power: the same 90 sessions now compare designs within a person rather than across people, and the error term shrinks by whatever the individual differences were worth. What you pay for is order: someone on their third interface is faster because they have solved two puzzles already, and possibly more bored. Practice, fatigue and carryover are all real, and none of them exist in a between-subjects design.
3. Counting the orders 3! = 3 × 2 × 1 = 6 possible orders With 30 participants you can run complete counterbalancing, five people per order, which balances practice effects across designs exactly. A Latin square using three sequences is the cheaper alternative when the number of conditions makes 6 impractical, and it balances first-order carryover rather than everything. Also vary the specific puzzles across designs so nobody sees the same problem twice.
4. Sphericity and ε Sphericity requires the variances of the differences between every pair of conditions to be equal. When it fails, the F-test's degrees of freedom are too generous and the p-value comes out too small. Greenhouse–Geisser multiplies both df by ε to compensate. uncorrected: F(2, 58) corrected: F(2 × 0.72, 58 × 0.72) = F(1.44, 41.76) Report the fractional degrees of freedom and name the correction. An ε of 0.72 is a meaningful departure, so with three conditions this is not optional.
5. When a mixed model wins Repeated-measures ANOVA needs a complete row for every participant, so one missing session throws that person out entirely. A mixed model keeps partial data, handles unequal numbers of trials per person, allows the covariance structure to be modeled rather than assumed, and extends to designs where the repeated factor is continuous. If sessions get canceled or the design grows a second random factor such as stimulus items, switch.

Run it within-subjects with a repeated-measures ANOVA, counterbalance across all 6 orders with 5 participants each, and report the Greenhouse–Geisser corrected test as F(1.44, 41.76).

From §1.5 Between vs Within Designs, §2.5 Repeated Measures ANOVA and §4.5 Mixed Models.

1. The right effect size Three groups compared with a one-way ANOVA means Cohen's f, not d. Cohen's benchmarks for f are .10 small, .25 medium, .40 large. Writing "medium effect" without naming the metric is the first thing to fix, because d = 0.5 and f = 0.25 are different quantities that happen to share the label.
2. Power at the planned n noncentrality λ = f²N = 0.25² × 60 = 3.75 df₁ = k − 1 = 2, df₂ = N − k = 57 power ≈ .37 If the effect she expects is really there, this design finds it about 37% of the time. She is more likely to end the year with an ambiguous null than with an answer, and a null from a study this size tells nobody anything.
3. The sample that works λ = 0.25² × 159 = 9.94, df₁ = 2, df₂ = 156 → power = .805 159 in total, so 53 per group. Set the power calculator to one-way ANOVA, f = 0.25, k = 3, α = .05, power = .80 and it returns the same figure. Two things to say in the write-up: "medium" is the weakest possible justification for an effect size, and the smallest effect that would actually change teaching practice is the better anchor. If prior studies are the source, discount them, because published effects run inflated.
4. Observed power Observed power is computed from the effect size she happens to obtain, which makes it a rearrangement of her own p-value rather than new information. A p just above .05 always yields observed power just under 50%, whatever the design. The question worth answering after a null result is different: what is the smallest effect this study could have detected with 80% power? Answer that from the design, before the data.
5. One more thing to warn her about She mentioned a possible follow-up with a second factor. Interactions typically need considerably more data than main effects of the same nominal size, because the interaction contrast is a difference of differences and its standard error is correspondingly larger. Power the study for the effect she most wants to detect, and if that is the interaction, plan the sample around it.

She needs Cohen's f = 0.25, not d. At 20 per group her power is about .37; reaching .80 takes 159 participants, 53 per group. Observed power after the fact adds nothing.

From §3.12 Power Analysis · run the numbers in Power & Sample Size or sketch the whole design in Plan My Analysis.

1. Tolerance and VIF tolerance = 1 − R²ⱼ VIF = 1 / tolerance

   Predictor	R²ⱼ	Tolerance	VIF
   Age	.923	.077	12.99
   Years of experience	.887	.113	8.85
   Years since qualifying	.861	.139	7.19

2. Reading the thresholds The common alarm level is VIF > 5, with VIF > 10 as the lenient version. All three predictors clear the strict threshold and age clears the lenient one. A tolerance of .077 says that only 7.7% of the variance in age is unique to age; the other 92.3% is already carried by the other two predictors.
3. Why the F and the t-tests disagree They are asking different questions. The F-test asks whether the three predictors together explain performance, and they clearly do: R² = .31. Each t-test asks whether one predictor adds anything once the other two are in the model, and since the three are nearly the same variable, none of them adds much on top of the others. This pattern is the textbook fingerprint of multicollinearity. The negative coefficient on experience is the same problem in a different costume: with predictors this entangled, the coefficient estimates become unstable, their standard errors inflate, and the fitted values can be reached by combinations that make no substantive sense. Do not interpret that minus sign as evidence that experience harms performance.
4. Three fixes and their costs Drop predictors and keep one: simplest, and it changes the question. The surviving coefficient is now the effect of that variable plus everything it stands in for, so name it honestly (say "career stage", not "age"). Combine them into a composite or a principal component: keeps all the information and buys interpretability problems, since a component score is no longer a thing anyone can act on. Regularize with ridge regression: stabilizes the estimates by accepting a little bias, which is the right call when prediction matters more than individual coefficients. There is also a fourth option worth naming, which is to collect data where the predictors come apart, for instance including career changers whose age and experience diverge.
5. Predictions are fine Multicollinearity inflates the variance of the coefficient estimates, not the fitted values. Within the range of predictor combinations actually observed, this model predicts as well as its R² suggests. What it cannot do is tell you which of the three variables is responsible, and that is usually the question that prompted the study.

VIFs of 12.99, 8.85 and 7.19, all above the usual alarm level of 5. The predictors are nearly the same variable, so the model as a whole is significant while no individual coefficient can be trusted, including its sign. Prediction survives; interpretation does not.

From §3.2 Multicollinearity · compare with the diagnostics in §2.10 and Problem 22 above.

1. Dropout by arm intervention: 18 / 90 = 20.0% control: 6 / 90 = 6.7% overall: 24 / 180 = 13.3% Differential dropout is a finding, not a nuisance. Three times as many people left the intervention arm, and the week-4 side-effect reports suggest why. A trial where the treatment drives people out has learned something important about the treatment.
2. Naming the mechanism Not MCAR: missingness is clearly related to arm, to baseline weight and to week-4 side effects, all of which are observed. That is the MAR pattern, and it is the workable case, because the observed variables carry the information needed to correct for it. What you cannot rule out is MNAR: if people quit precisely because the scale was not moving, missingness depends on the unobserved week-12 weight itself, and no amount of cleverness with the observed data settles that. The distinction is untestable from the data. It is an assumption you argue for and then probe.
3. Direction of the bias Complete-case analysis keeps the people who stayed, and in the intervention arm those are disproportionately the ones for whom it was working and tolerable. The 20% who left were heavier at baseline and struggling with side effects. So the −5.8 kg is an estimate of what the treatment does for people who can stick with it, which flatters it relative to what it does for everyone offered it. The control arm loses far fewer people, so the comparison is unbalanced on top of being optimistic.
4. Three fixes, graded Mean imputation is the worst of the three. It inserts values with zero variability, shrinks the standard deviation, weakens every correlation, and leaves the software believing it has a full 180 cases, so the standard errors and p-values come out overconfident. Last observation carried forward is better than it looks in this specific trial, since carrying a week-4 weight forward in a study where weight is falling understates the intervention's benefit and is therefore conservative. It is still an assumption dressed up as data, it does not propagate any uncertainty, and regulators have moved away from it. Multiple imputation is the right tool: build the imputation model from baseline weight, arm, week-4 weight and side-effect reports, generate several completed datasets, analyze each, and pool. The extra uncertainty from the holes then flows honestly into the standard errors. A mixed model fitted to all timepoints reaches the same place under MAR and needs no imputation step at all.
5. What the paper should say How much was missing and from which arm; a comparison of dropouts and completers on every baseline variable available; the mechanism assumed and the reasoning behind it; the primary analysis under MAR, by multiple imputation or a mixed model; and a sensitivity analysis that asks what happens if the dropouts did badly, for instance assuming they returned to baseline weight. If the conclusion survives that, say so. If it does not, that is the result.

20% dropout in the intervention arm against 6.7% in the control arm, related to observed variables, so MAR at best and possibly MNAR. Complete-case analysis keeps the people it worked for and overstates the benefit. Use multiple imputation or a mixed model, and report a sensitivity analysis alongside it.

From §4.9 Missing Data · the reporting standard belongs to §3.4 From Output to Results.

1. Natural frequencies, 100,000 people

   	Test positive	Test negative	Total
   Has the condition	760	40	800
   Does not have it	7,936	91,264	99,200
   Total	8,696	91,304	100,000

   800 × .95 = 760 true positives 99,200 × .08 = 7,936 false positives Counting people instead of manipulating probabilities is the single most reliable way to get this right, and it is how the answer becomes obvious rather than surprising.
2. The answer she wants P(condition | positive) = 760 / 8,696 = .0874 About 9%. Of every 100 positive results the program issues, roughly 91 are false alarms. The test has not failed; it is doing exactly what its specificity promises. The trouble is that 8% of a very large healthy group outnumbers 95% of a very small sick one.
3. A second positive posterior odds = prior odds × likelihood ratio LR⁺ = sensitivity / (1 − specificity) = .95 / .08 = 11.875 prior odds = .008 / .992 = .00806 after one positive: .00806 × 11.875 = .0958 → P = .0958/1.0958 = .0874 ✓ after two positives: .0958 × 11.875 = 1.1372 → P = 1.1372/2.1372 = .532 The first test moved her from 0.8% to 8.7%, and the second from 8.7% to 53%. Yesterday's posterior is today's prior, and the odds form makes the update a single multiplication. The word "independent" is carrying real weight here: if both tests share a systematic quirk, for instance both cross-reacting with the same protein, the second result adds far less than this calculation assumes.
4. A negative result P(clear | negative) = 91,264 / 91,304 = .9996 99.96%. When the condition is rare, a negative result is extremely informative and a positive one barely is. Screening tests are usually built this way on purpose, because the positives get followed up and the negatives get sent home.
5. Why the intuition fails Sensitivity answers "if you are sick, will the test say so?" She is asking the reverse: "the test said so, am I sick?" Those two conditional probabilities are only equal when the base rate is 50%, and here it is 0.8%. Confusing them is base-rate neglect, and it is not a beginner's mistake. Doctors asked this exact question in studies of medical decision-making get it wrong at high rates too, until the numbers are presented as counts of people.

A single positive leaves her at about 9%, two independent positives at 53%, and a negative result clears her with 99.96% probability. Nothing is wrong with the test; the base rate is doing the work.

From §4.2 Bayesian Thinking · the same 2 × 2 is the confusion matrix in §5.4 of the ML course.

1. The update Beta(a, b) prior + k successes in n trials → Beta(a + k, b + n − k) = Beta(8 + 33, 12 + 27) = Beta(41, 39) The Beta is conjugate to the binomial, which is why the update is addition rather than integration. A Beta(a, b) prior is worth a + b − 2 pretend observations, so this one carried the weight of 18 previous users.
2. Summarizing the posterior mean = 41 / 80 = .5125 mode = 40 / 78 = .5128 SD = √[(41 × 39) / (80² × 81)] = √(1599 / 518400) = √.003085 = .0555 Mean and mode agree to three decimals because the posterior is nearly symmetric at these values. They part company when a or b is small.
3. Why .5125 and not .55 The prior mean was 8/20 = .40, and the prior brought 18 pseudo-observations to a dataset of 60. The posterior lands between the two, pulled toward the prior in proportion to how much evidence each side carries. With a flat Beta(1, 1) prior instead, the posterior would be Beta(34, 28) with mean .548, essentially the raw .55. Which of those two answers is right depends on a question the arithmetic cannot settle: if last quarter's users really are informative about this quarter's, the shrinkage is information rather than distortion, and if they are not, that prior should not have been used.
4. P(θ > .50) and the interval z = (.50 − .5125) / .0555 = −.225 P(θ > .50) ≈ 1 − Φ(−.225) = .589 The exact posterior probability is .5889, so the normal approximation is safe here. About a 59% chance the new flow beats a coin flip, which is a much more honest summary than "33 out of 60 is over half". The interval in words: given the prior and these 60 users, there is a 95% probability that the true completion rate lies between .404 and .621. That sentence is legal for a credible interval and illegal for a confidence interval, which is most of the reason people reach for Bayesian methods.
5. What to tell the team The interval spans .404 to .621, so the data do not yet distinguish a mild improvement from a mild regression. 60 users is not enough. Rerun the numbers after a few hundred and the interval will tighten around whatever is true.

Posterior Beta(41, 39): mean .5125, mode .5128, SD .0555. The prior's 18 pseudo-observations pull the estimate below the observed .55, and P(θ > .50) is about .59.

From §4.3 Bayesian Estimation, building on §4.2.

1. Participant A H = 34/40 = .85 FA = 12/40 = .30 d′ = z(H) − z(FA) = 1.036 − (−0.524) = 1.56 c = −[z(H) + z(FA)] / 2 = −[1.036 − 0.524] / 2 = −0.26 A negative c is a liberal criterion: A says "old" more readily than a neutral observer would, which buys hits and pays for them in false alarms.
2. Bias as a likelihood ratio, and the ROC area β = e^c·d′ = e^{(−0.256)(1.561)} = e^−0.400 = 0.67 AUC = Φ(d′/√2) = Φ(1.561 / 1.414) = Φ(1.104) = .865 β below 1 says the same thing as negative c: at A's criterion the evidence is not yet more likely under "old" than under "new", and A commits anyway. The AUC of .865 is the bias-free summary. It would be identical for an observer with A's sensitivity and any criterion whatsoever.
3. Participant C, and the percent-correct trap H = 38/40 = .95 FA = 20/40 = .50 d′ = 1.645 − 0.000 = 1.64 c = −[1.645 + 0]/2 = −0.82 A's percent correct = (34 + 28)/80 = .775 C's percent correct = (38 + 20)/80 = .725 C scores five points worse on percent correct and yet has slightly higher sensitivity, 1.64 against 1.56. The whole difference is strategy: C is far more liberal (c = −0.82) and pays for a near-perfect hit rate with 20 false alarms. Percent correct blends skill and strategy into one number and then hides which one moved. That is the argument for reporting d′ and c separately, and it is why a memory drug that "improves recognition" means nothing until the false-alarm rate is on the table.
4. Participant B and the ceiling H = 40/40 = 1.00, and z(1) is infinite, so d′ cannot be computed. The standard repair is the log-linear correction: add 0.5 to every cell count and 1 to every trial total, applied to every participant rather than only to the offending one. H = (40 + 0.5)/(40 + 1) = 40.5/41 = .988 FA = (8 + 0.5)/41 = 8.5/41 = .207 d′ = 2.251 − (−0.816) = 3.07 c = −[2.251 − 0.816]/2 = −0.72 Applying the correction to everyone keeps the estimates comparable; correcting only the extreme participants would bias that person relative to the rest. Name whichever rule you used in the methods section, and treat a d′ above 3 with some caution, because rates this close to the ceiling make the estimate fragile.

A: d′ = 1.56, c = −0.26, β = 0.67, AUC = .865. C: d′ = 1.64 with a much more liberal c = −0.82, which is why C has the higher sensitivity and the lower percent correct. B needs the log-linear correction, giving d′ = 3.07.

From §4.12 Signal Detection Theory · the ROC half of it is §5.5 in ML & AI.

1. PSE and JND quiet: PSE = 3.60 / 0.0090 = 400.0 ms quiet: JND = 1.0986 / 0.0090 = 122.1 ms loud: PSE = 2.94 / 0.0084 = 350.0 ms loud: JND = 1.0986 / 0.0084 = 130.8 ms The PSE is where the fitted curve crosses 50%, so it is the duration that feels exactly midway between the anchors. The JND is half the distance between the 25% and 75% points, which is the curve's steepness expressed in milliseconds.
2. Weber fractions quiet: 122.1 / 400.0 = .305 loud: 130.8 / 350.0 = .374 Dividing precision by the point it was measured at makes the two conditions comparable despite sitting at different places on the duration axis.
3. Checking the JND from the curve 75% point: b₀ + b₁x = ln 3 → x = (1.0986 + 3.60)/0.0090 = 522.1 ms 25% point: b₀ + b₁x = −ln 3 → x = (−1.0986 + 3.60)/0.0090 = 277.9 ms half the span = (522.1 − 277.9)/2 = 122.1 ms ✓ The ln 3 shortcut comes straight from the logistic: at 75% the odds are 3 to 1, so the linear predictor equals ln 3.
4. Shift or slope Mostly a shift. The PSE drops by 50 ms while the JND rises by only 8.7 ms, so the curve slid left without flattening much. Louder probes are judged longer, which is a bias in perceived duration rather than a loss of timing precision. Had the JND doubled with the PSE unmoved, the finding would have been the opposite kind: the same subjective midpoint, measured with a noisier instrument. In a regression this dissociation is two terms. The condition main effect tests the shift, and the condition × duration interaction tests the slope change.
5. 400 rather than 500 500 ms is the arithmetic midpoint of 200 and 800. 400 ms is their geometric mean, √(200 × 800), and bisection points landing near the geometric mean is one of the standard findings in timing research. It suggests the internal scale for duration is compressive, so the subjective middle sits below the arithmetic one. Getting 400 here is a signature, not an error.

Quiet: PSE 400.0 ms, JND 122.1 ms, Weber .305. Loud: PSE 350.0 ms, JND 130.8 ms, Weber .374. A 50 ms leftward shift with almost no change in slope, so louder tones feel longer without being harder to time.

From §4.11 Psychometric Functions · the fitting machinery is §3.7 Logistic Regression.

1. Families and links

   Outcome	Family · link	eb means
   a. Attacks per month	Poisson · log (negative binomial if overdispersed)	Rate ratio
   b. Readmitted yes/no	Binomial · logit	Odds ratio
   c. Cost in euros	Gamma · log	Multiplicative change in mean cost
   d. Recalled out of 20	Binomial · logit, as successes out of 20	Odds ratio per item
   e. Blood pressure	Gaussian · identity (ordinary regression)	Nothing; read b directly

   Two are worth pausing on. Outcome (d) looks like a count but has a known ceiling of 20, which makes it a binomial proportion rather than a Poisson count; modeling the raw proportion with ordinary regression would happily predict recall of 23 items. Outcome (c) is the classic case for Gamma: costs cannot go negative, and their variance grows with their mean, which is exactly the mean-variance relationship Gamma encodes.
2. Interpreting the drug coefficient rate ratio = e^b₁ = e^−0.51 = 0.60 placebo rate = e^b₀ = e^0.92 = 2.51 attacks per month drug rate = e^{b₀ + b₁} = e^0.41 = 1.51 attacks per month check: 2.51 × 0.60 = 1.51 ✓ The drug multiplies the attack rate by 0.60, a reduction of about 40%. Coefficients are additive on the log scale and multiplicative on the count scale, which is the same pattern as odds ratios in logistic regression.
3. Overdispersion deviance / df = 412.6 / 178 = 2.32 A Poisson model assumes variance equals mean, which puts this ratio near 1. At 2.32 the counts vary more than twice as much as Poisson allows, and the consequence is standard errors that are too small and p-values that are too impressive. Migraine counts cluster within people, so this is the expected outcome rather than a surprise. Refit as a negative binomial, which adds its own dispersion parameter, or use quasi-Poisson to rescale the standard errors. The point estimate of the rate ratio will barely move; the confidence interval around it will widen, correctly.

Poisson-log, binomial-logit, Gamma-log, binomial-logit again, and plain Gaussian. The drug cuts the attack rate from 2.51 to 1.51 per month, a rate ratio of 0.60, but a deviance-to-df ratio of 2.32 means the Poisson standard errors are not trustworthy until the model is refitted as negative binomial.

From §4.4 Generalized Linear Models · the logit branch is §3.7.

1. The Kaplan–Meier table S(t) = Π (1 − dⱼ / nⱼ) over all event times up to t

   t	At risk	Events	1 − d/n	S(t)
   3	10	1	.900	.9000
   5+	9	0	censored	.9000
   7	8	1	.875	.7875
   9	7	2	.714	.5625
   12+	5	0	censored	.5625
   14	4	1	.750	.4219
   18+	3	0	censored	.4219
   20	2	1	.500	.2109
   24+	1	0	censored	.2109

   Two things drive the whole table. Censored patients count in the at-risk column right up to the moment they leave, and then simply drop out of it. The at-risk number falls by both events and censorings, which is why only 4 people are at risk at month 14 even though 6 have not yet relapsed on paper.
2. Two readings off the curve S(12) = .5625 S(15) = .4219 Nothing happens at month 12 except a censoring, so the curve is flat across it and S(12) is still the value set at month 9. The staircase only steps down at event times.
3. Median survival The median is the first time at which S(t) drops to .50 or below. At month 9 the curve is at .5625, still above half. At month 14 it reaches .4219. Median survival = 14 months. Read it off the vertical axis at 0.5 and drop to the horizontal one; do not average the observed times.
4. The two bad averages all ten times: (3+5+7+9+9+12+14+18+20+24)/10 = 121/10 = 12.1 months events only: (3+7+9+9+14+20)/6 = 62/6 = 10.3 months The first treats every censored time as though the patient relapsed at that moment, which understates survival, since we only know they lasted at least that long. The second is worse: throwing out censored cases deletes precisely the people who were doing well, so it drops from 12.1 to 10.3 and moves further from the truth. The Kaplan–Meier median of 14 months is above both, because it is the only one of the three that uses the censored patients' partial information instead of destroying it.
5. Why it stops at .2109 The longest observation, at 24 months, is censored. The curve has no information about what happens after that point, so it simply stops rather than pretending everyone eventually relapses. Estimates from the flat tail of a survival curve rest on very few people, which is why a curve that ends with 1 or 2 patients at risk should be read with the confidence band, not the line.

S steps to .9000, .7875, .5625, .4219 and .2109, giving a median survival of 14 months and S(12) = .5625, S(15) = .4219. Both naive averages (12.1 and 10.3 months) understate survival, and the curve ends at .2109 because the last patient was censored.

From §4.8 Survival Analysis.

1. AIC for each model M1: −2(−455.2) + 2(3) = 910.4 + 6 = 916.4 M2: −2(−449.8) + 2(4) = 899.6 + 8 = 907.6 M3: −2(−449.1) + 2(5) = 898.2 + 10 = 908.2 M4: −2(−447.3) + 2(8) = 894.6 + 16 = 910.6 Ranking, best first: M2, M3, M4, M1. Lower is better, and the numbers themselves mean nothing in isolation. Only differences between models fitted to the same data are interpretable.
2. ΔAIC ΔM2 = 0.0 ΔM3 = 0.6 ΔM4 = 3.0 ΔM1 = 8.8 The common reading: Δ under about 2 means the models are effectively indistinguishable, 4 to 7 means considerably less support, and above 10 means essentially none. So M2 and M3 are tied, M4 is trailing but not dead, and M1 is clearly out. The honest summary is that this dataset cannot separate M2 from M3.
3. Akaike weights and the evidence ratio relative likelihood = e^−Δ/2 M1: e^−4.4 = 0.012 M2: e⁰ = 1.000 M3: e^−0.3 = 0.741 M4: e^−1.5 = 0.223 sum = 1.976 w(M2) = 1.000/1.976 = .506 w(M3) = 0.741/1.976 = .375 evidence ratio = .506/.375 = 1.35 = e^0.3 M2 is 1.35 times better supported than M3. That is close to nothing. For comparison, M2 over M1 is e^4.4 = 81 times better supported, which is what a real difference looks like.
4. The nested test χ² = 2(ln L₃ − ln L₂) = 2(−449.1 + 449.8) = 2(0.7) = 1.4 df = 5 − 4 = 1, p = .24 Motivation does not earn its parameter. The likelihood-ratio test and AIC agree, which is what usually happens with one extra parameter, since AIC charges 2 and the χ² critical value at df = 1 is 3.84. They can disagree in either direction as models grow.
5. Why the best fit loses, and what BIC says BIC: M1 924.8 M2 918.7 M3 922.1 M4 932.9 M4's log-likelihood is the highest because adding parameters can never make in-sample fit worse; setting a new coefficient to zero is always available, so the fit can only improve or stay put. AIC charges 2 per parameter for exactly this reason, and M4's four extra parameters bought only 2.5 units of log-likelihood, which is not enough. BIC picks M2 as well and does so more decisively, because its penalty of ln 120 = 4.79 per parameter is more than twice AIC's. Agreement between the two is reassuring; the interesting case is when they disagree, which usually means a moderately sized effect is on the edge of paying for itself. What would actually settle M2 against M3 is not another criterion but new data, or cross-validation on the data at hand.

AIC = 916.4, 907.6, 908.2 and 910.6, so M2 wins with M3 only ΔAIC = 0.6 behind, an evidence ratio of 1.35 that settles nothing. The likelihood-ratio test agrees that motivation is not worth its parameter (χ²(1) = 1.4, p = .24), and BIC picks M2 more decisively.

From §3.9 Model Comparison · the held-out version is §4.6 Cross-Validation.

1. Self-selection Nobody assigned anyone to anything. Students decided for themselves whether to use the module, so the two groups differ in whatever made them decide, and the module is only one candidate explanation for the nine-point gap. The t-test is valid as a description of two groups that already existed. It is the word "raises" that has no support, because a comparison of self-selected groups cannot distinguish an effect of the treatment from a difference between the people who chose it.
2. Three plausible confounds Prior mathematical preparation: students who already find statistics tractable are more likely to engage with extra material and more likely to score well. Conscientiousness: the trait that produces module use also produces lecture attendance, problem sets and revision. Available time: students working long hours in paid jobs have less of everything, including both module use and study time. Each of these causes both the exposure and the outcome, which is the definition of a confound.
3. Does controlling for GPA fix it? It helps and it does not fix it. Adjusting for prior GPA removes the part of the gap explained by measured prior ability, which is real progress, and the adjusted estimate will almost certainly be smaller than nine points. What remains is everything that predicts both usage and outcome but was never measured: motivation this term, hours available, whether a friend recommended the module. Statistical adjustment can only remove confounding you have data on. This is where the language of causal diagrams earns its keep, because drawing the assumed structure makes visible which variables you would need and which you actually have.
4. The study that would settle it Randomize access. Give the module to a random half of the cohort and withhold it from the other half for the term, or stagger the rollout so every student eventually gets it but the order is random, which is usually easier to get past an ethics committee and a students' union. Pre-register the primary outcome and the analysis. Analyze by assigned group rather than by actual usage, because the moment you analyze by who chose to log in, self-selection is back. If randomization is genuinely impossible, a regression-discontinuity or difference-in-differences design around a rollout date beats a naive comparison.
5. An honest headline "Students who chose to use the new module scored nine points higher, though they may have differed before the term began." Longer, duller, and true. The internal version should add that a randomized rollout next year would answer the question properly.

Self-selection. The groups were formed by student choice, so the nine-point gap confounds the module with prior ability, conscientiousness and available time. Controlling for GPA shrinks the estimate without licensing causal language; only random assignment does that.

From §1.4 Experiments & Random Assignment and §1.6 Quasi-Experiments.

1. The frame and the volunteers Two filters run before anyone answers. Coverage: only people who visit the council website or follow its social accounts ever see the link, which excludes residents without reliable internet, those who do not follow local government, and anyone whose news comes through other channels. Self-selection: of the people who saw it, the ones who answered are those who cared enough to click, and strong feelings on both sides drive response far more than mild ones. Organized campaigns on either side can mobilize supporters through a single post. The realized sample is people who saw the link and chose to respond, which is not the adult population of the city.
2. Why 4,300 does not help Sample size fixes variance, not bias. A convenience sample's estimate is centered in the wrong place, and collecting more of the same kind of response tightens the cloud around that wrong center. 4,300 self-selected responses give a more precise wrong answer than 430 would. The only cure for bias is a better sampling method.
3. The margin of error The ±1.5% figure is the most misleading number in the report. That formula assumes a probability sample in which every resident had a known, non-zero chance of selection, and here nobody was selected at all; people arrived. Quoting it attaches the authority of survey statistics to a number that has none of its properties. It describes sampling error alone, and sampling error is not this survey's problem.
4. One more threat Nothing stops one person responding repeatedly, or a campaign asking supporters to. An open link with no authentication has no way to enforce one response per resident, and no way to confirm that respondents live in the city at all.
5. What to do, and what the survey is worth For an estimate: draw a probability sample from the electoral register, stratify by neighborhood and age band so every part of the city is represented, follow up non-responders more than once, and report the response rate alongside the result. That design supports both the percentage and its margin of error. The open survey is not worthless, though. As a consultation it surfaces arguments, local knowledge and objections the council would not otherwise hear. It should be reported as what it is: 4,300 people chose to tell us this, with no claim about the other 255,700.

A self-selected sample drawn from an incomplete frame. The 78% describes people who saw the link and chose to respond, the sample size makes that no more trustworthy, and the ±1.5% margin of error is meaningless because the formula assumes probability sampling that never happened.

From §1.8 Sampling Methods and §1.9 Designing Surveys.

1. Post-hoc exclusion "After inspecting the data we excluded 7 participants" is an outlier rule chosen after seeing what it would do, and a 2 SD cutoff is lenient enough to remove roughly 5% of a normal distribution by construction. Calling the excluded responses "clearly unrepresentative" is a judgment made with knowledge of the result. Fix: specify the exclusion rule before data collection, apply it blind to condition, and report the analysis both with and without exclusions.
2. Optional stopping "Because the effect was not significant, we collected a further 30 participants" is the textbook definition of data-dependent stopping. Every additional look is another chance for the p-value to wander below .05, and this practice alone can push a nominal 5% false-positive rate toward 20% or more. Fix: fix the sample size in advance from a power analysis, or use a pre-planned sequential design with corrected thresholds, which is a legitimate and sometimes required approach. What is not legitimate is deciding after each look.
3. Selective reporting of outcomes Four measures were collected and one is reported. The reader has no way to know whether reaction time was the planned primary outcome or the one that survived four attempts. Fix: report every measure collected, nominate a primary outcome in advance, and correct for multiplicity across the rest or label them exploratory.
4. HARKing "As predicted" cannot be assessed by anyone, and given the three practices above there is good reason to doubt it. Presenting a finding located by exploration as a prediction converts a hypothesis-generating result into apparent confirmation. Fix: separate confirmatory from exploratory sections explicitly, and let a preregistration carry the claim about what was predicted.
5. The degrees of freedom 60 recruited − 7 excluded + 30 added = 83 participants two groups, so df_error = 83 − 2 = 81 ✓ The arithmetic is fine. Every number in the paragraph is internally consistent, which is exactly why this manuscript would pass a statistical-consistency screen and still be untrustworthy. The problem is the path taken to those numbers, and no recomputation can detect it.
6. Why this outranks fraud Fabrication is rare, with under 1% of surveyed researchers admitting it, and it is unambiguous when caught. These practices are common: in a survey of more than 2,000 psychologists, around 63% admitted failing to report all dependent measures and about 56% admitted deciding whether to collect more data after checking significance. They also feel like reasonable judgment calls at the time, which is why they persist. Rare deliberate fraud damages a few papers; widespread gray-zone flexibility fills the literature with effects that evaporate on replication, and that is the mechanism behind the replication crisis.

Post-hoc outlier exclusion, optional stopping, selective reporting of one outcome out of four, and HARKing. The degrees of freedom check out perfectly, which is the point: consistent numbers are no defense against a flexible analysis path.

From §4.5 Questionable Research Practices, §1.11 The Replication Crisis and §1.12 Preregistration.

1. Step-by-step verdict

   Step	Verdict	Why
   a. Listwise deletion	Defensible, needs justification	Fine if the data are plausibly MCAR and the loss is small, but it must be reported and it contradicts step (b)
   b. Blanks → 0	Error	0 is not a valid response on a 1–7 scale
   c. 999 left in	Error	999 is a missing-value code, not an age
   d. Reverse by ×(−1)	Error	Produces −1 to −7, destroying the scale
   e. Dropped lowest scorers	Serious error	Selection on the outcome variable
   f. Merged on initials	Error	Collides, breaks on name changes, and is identifying
   g. Overwrote the raw file	Serious error	The original data no longer exists

2. Fixing (b) and (d) For (b), leave missing values missing and decide an item-level rule instead: compute the scale when at least 80% of its items were answered, prorating from the items that were, and set it to missing otherwise. Filling with 0 invents the most extreme response available and drags every affected participant's total toward the floor. correct reverse-scoring: x' = (min + max) − x = 8 − x so 1 → 7, 2 → 6, 4 → 4, 7 → 1 Multiplying by −1 gives a range of −1 to −7, which is not the reversed scale, and it corrupts every scale total that includes those three items without producing a single value that looks obviously wrong. Recompute the reversed items from the raw file, which is a problem given step (g).
3. The true mean age total of all 240 ages = 240 × 33.2 = 7,968 the three sentinels contribute 3 × 999 = 2,997 remaining total = 7,968 − 2,997 = 4,971 true mean = 4,971 / 237 = 21.0 years Three cells inflated the reported mean by 12.2 years in a sample of 240. This is what missing-value sentinels do when a pipeline treats them as data, and it is why the first step of any cleaning workflow is to check every variable's minimum and maximum against its plausible range.
4. The most serious step Step (e), with step (g) close behind. Removing the six lowest wellbeing scores is not cleaning; it is deleting cases on the basis of the outcome variable, which changes the mean by construction and does so in a direction the researcher can see while doing it. If non-serious responding is a genuine concern, screen for it with something independent of the outcome: attention-check items, implausibly fast completion times, straight-lining down a column. Step (g) is a different kind of damage. Raw data should be read-only and every transformation should live in a script, so that the cleaned file can always be regenerated and any mistake in this log can be undone. As things stand, none of these errors can be reversed.

Five clear errors: 0 for missing on a 1–7 scale, a 999 sentinel treated as an age (which inflated the mean from 21.0 to 33.2), reverse-scoring by −1 instead of 8 − x, dropping the lowest scorers, and merging on initials. Overwriting the raw file is what makes all of them permanent.

From §2.4 The Cleaning Workflow, §2.5 Outliers and §2.9 Reproducible Workflows · the worked version is the survey-cleaning guide.

1. The catalog of problems The truncated axis is the big one: starting at 7.2 instead of 0 turns a 0.27-point gap into a bar nearly three times the height of its neighbor, since A occupies 25% of the plot area and B occupies 70%. Bars encode value by length, so a bar chart with a truncated axis is misreporting its own data. Beyond that: no uncertainty is shown, so a reader cannot tell a solid difference from noise; the 3-D perspective distorts the apparent heights and adds nothing; two bars summarize 79 people and hide every distribution; and the caption asserts a conclusion ("clearly outperforms") that the figure does not establish and, as question 3 shows, is wrong.
2. Where the 27% came from 7.62 − 7.35 = 0.27 points relative change = 0.27 / 7.35 = 3.7% Someone read the difference of 0.27 as "27%". The actual relative increase in the mean is 3.7%, and even that is a shaky thing to report, because a 1–10 satisfaction scale has no meaningful zero, so percentage changes on it depend on an arbitrary origin. The honest statement is the difference itself: Layout B scored 0.27 points higher on a 10-point scale.
3. Is it distinguishable from zero? s²_pooled = [40(1.42²) + 37(1.38²)] / 77 = (80.66 + 70.46)/77 = 1.963 s_pooled = 1.401 SE = 1.401 × √(1/41 + 1/38) = 1.401 × .2252 = 0.3155 t(77) = 0.27 / 0.3155 = 0.86, p = .39 d = 0.27 / 1.401 = 0.19 No. A difference this size arises about 39% of the time when the two layouts are identical. The figure's entire visual argument rests on a difference the data cannot distinguish from nothing, and the two-million rollout has no evidence behind it. With d = 0.19, detecting an effect this small with 80% power would need over 400 stores per layout rather than 40.
4. The figure to send instead Plot the individual store scores as two columns of points, jittered so they do not overlap, with the group mean marked and a 95% confidence interval drawn around it. Run the y-axis across the full 1 to 10 scale, or across a clearly labeled subrange with the truncation stated in the caption. Drop the third dimension. Make the caption descriptive: "Figure 2. Mean customer satisfaction by store layout, with 95% confidence intervals. Individual stores shown as points; n = 41 and 38." Then let the overlapping intervals say what they say. A figure whose job is to persuade is not a figure, and if the recommendation is to keep testing, the figure should make that obvious at a glance.

Truncated axis, no error bars, gratuitous 3-D, a conclusion in the caption, and a "+27%" that is really a 0.27-point difference (3.7% in relative terms). The difference is not distinguishable from zero, t(77) = 0.86, p = .39, d = 0.19, so the rollout rests on nothing.

From §3.3 Tables & Figures That Don't Lie · check the size of the effect in the effect-size converter.

1. Error 1: a leading zero on p written: p = 0.062 correct: p = .062 Drop the leading zero from any statistic that cannot exceed 1 in magnitude: p, r, R², η², β and Cramér's V. A probability of 0.062 is never going to be 1.062, so the zero carries no information.
2. Error 2: a missing zero on d written: d = .49 correct: d = 0.49 The mirror image of the first rule, and the one people miss because they over-apply the first. Cohen's d can comfortably exceed 1, so it keeps its leading zero. The same goes for t, F, M and SD. The test is not "is it a decimal" but "can this quantity exceed 1".
3. Error 3: the wrong df on the correlation written: r(60) = .34 correct: r(58) = .34 The number in parentheses after r is degrees of freedom, not sample size, and for a correlation that is n − 2. With 60 participants the value is 58. Writing r(60) tells a careful reader you had 62 people, which contradicts the methods section. The t-test in the same sentence has it right, and the two matching by coincidence here is a good reminder to check rather than pattern-match.
4. The corrected sentence "Participants receiving immediate feedback (M = 78.4, SD = 8.6) scored higher on accuracy than those receiving delayed feedback (M = 74.1, SD = 8.9), though the difference was not statistically significant, t(58) = 1.90, p = .062, d = 0.49. Accuracy correlated with practice time, r(58) = .34, p = .008."
5. Checking the numbers SE = √[(8.6² + 8.9²)/30] = √(153.17/30) = 2.26 t = (78.4 − 74.1)/2.26 = 1.90 ✓, two-tailed p = .062 ✓ s_pooled = √[(8.6² + 8.9²)/2] = 8.75, so d = 4.3/8.75 = 0.49 ✓ for r = .34 with df = 58: t = .34√(58/(1 − .34²)) = 2.75, p = .008 ✓ Everything reconciles, which is what the consistency check in the APA formatter does automatically. Formatting errors and arithmetic errors are separate species, and a sentence can have either without the other.
6. What the supervisor circled "Were more accurate" states a difference at p = .062. That is not a formatting error; it is a claim the test does not support. The correction above replaces it with a description of the observed means plus an explicit statement that the difference was not significant, which is the honest form. Resist both temptations here: do not write "approached significance", and do not pretend the 4.3-point gap was not observed. Report the estimate, report that it is compatible with no difference at this sample size, and let the confidence interval carry the uncertainty.

Leading zero on p (should be .062), missing leading zero on d (should be 0.49), and r(60) where the degrees of freedom are n − 2 = 58. The numbers themselves all reconcile; the fourth problem, calling p = .062 a difference, is a writing error rather than a formatting one.

From §3.2 Reporting Statistics in APA Style and §3.5 Non-Significant Results · the rules are on the APA cheat sheet and applied by the APA formatter.