Signal Detection Theory

A radiologist stares at a chest X-ray. Somewhere in the gray there is either a tumor or nothing, and the image refuses to say which: diseased tissue sometimes photographs clean, healthy tissue sometimes looks suspicious. Signal detection theory is the statistics of decisions made on evidence that overlaps like that. Its gift to research is a clean separation between two things every raw accuracy score blurs together: how well the observer can tell the two worlds apart (sensitivity), and how willing they are to say “yes” (response bias).

One question, four outcomes

The basic experiment could not be simpler. On every trial something is either there (signal) or not (noise), the observer answers yes or no, and each trial lands in one of four cells. A yes to a real signal is a hit; a no is a miss. A yes when nothing was there is a false alarm; a no is a correct rejection. If that table feels familiar, it should: it is the same 2×2 you met as the confusion matrix, with a human playing the role of the classifier.

Because each row must sum to 100%, the whole table boils down to two independent numbers: the hit rate H (the share of signal trials answered yes) and the false-alarm rate FA (the share of noise trials answered yes). And two numbers is one more than a percent-correct score keeps. That lost number is the interesting one. An observer who answers yes on every single trial earns a perfect 100% hit rate, alongside a 100% false-alarm rate; they detected nothing at all. Accuracy alone cannot tell that strategy apart from skill.

Two bells and one line

The model behind the theory is almost drawably simple. Every trial produces an internal evidence value in the observer's head: how tumor-ish the X-ray looks, how word-like the memory feels. Noise trials generate evidence scattered around a low average; signal trials generate evidence from the same bell shifted to the right. The distance between the two bells, measured in standard deviation units, is d′ (“d-prime”): the observer's sensitivity. Far-apart bells mean the two worlds rarely produce confusable evidence; overlapping bells mean frequent confusion no strategy can remove.

The observer's decision rule is a single vertical line, the criterion: answer yes whenever the evidence lands to its right. Placed low (to the left), the rule is liberal, producing many hits and many false alarms; placed high, it is conservative, producing few of both. The statistic c measures that placement as the distance from the neutral point halfway between the bells, so c = 0 is unbiased, positive is conservative, negative is liberal.

Everything is computed with the z-scores you already know. Convert the two observed rates into normal quantiles and both parameters fall out:

d′ = z(H) − z(FA)      c = −[z(H) + z(FA)] / 2

A third index, β = e^c·d′, expresses the same bias as a likelihood ratio: how many times more probable the evidence at the criterion must be under signal than under noise before the observer commits to yes. Play with all of it below.

Skill or strategy: two ways to get more hits

The playground makes the central lesson physical. There are exactly two ways to raise a hit rate. Pull the bells apart (raise d′), and hits climb while false alarms fall: the observer genuinely distinguishes better. Or slide the criterion left (lower c), and hits climb while false alarms climb with them: nothing about the perception improved, only the willingness to commit. Only the first is skill. A memory drug that “improves recognition by 10%” has shown nothing until the false-alarm rate is on the table too.

Neither parameter is more “real” than the other, though. Criterion placement is where costs, rewards, and base rates enter the decision. A smoke detector is engineered to be extremely liberal, because a false alarm costs a minute of annoyance while a miss costs a house. A criminal jury is instructed to be conservative for the mirror-image reason. Airport screeners, spam filters, and drowsy radiologists all live at some chosen point on this trade-off, and signal detection theory is the language for saying where, separately from how good they are.

Sweep the criterion and you draw the ROC

Every criterion placement yields one (false-alarm rate, hit rate) pair: a single point in the square panel above. Sliding the criterion across the whole evidence axis drags that point along a curve, from the ultra-conservative corner (never say yes: 0, 0) to the ultra-liberal one (always say yes: 1, 1). That curve is the receiver operating characteristic, and the “receiver” in the name is a 1950s radar operator judging blips, which is where this theory was born. The ROC curve you met in the ML course is the identical object with a model's score threshold standing in for the observer's criterion.

The curve's shape depends on d′ alone. Moving the criterion slides the operating point along the curve; only a change in sensitivity lifts the whole curve toward the top-left corner. That makes ROC area the bias-proof summary of performance: for the equal-variance model, AUC = Φ(d′/√2), so d′ = 1 gives 0.76 and d′ = 2 gives 0.92, the same numbers the ML lesson reports for its separation slider. Memory researchers exploit this daily: asking participants for confidence ratings instead of a bare yes/no yields several criteria at once, an empirical ROC, and a sensitivity estimate no criterion shift can fake.

From counts to d′

Real data arrive as four counts, and the arithmetic is two z-transforms long. Say a recognition-memory test used 20 old and 20 new words, and a participant produced 15 hits and 4 false alarms. Then H = .75 and FA = .20, so z(.75) = 0.674 and z(.20) = −0.842, giving d′ = 0.674 − (−0.842) = 1.52 and c = −(0.674 − 0.842)/2 = 0.08: solid discrimination, close to neutral bias. The “Try it yourself” box below runs the same four lines in R or Python.

Beyond the lab, the same two-number logic runs through recognition-memory experiments (old/new judgments), audiology and vision screening, radiologists' diagnostic performance, eyewitness lineups, and quality inspection. And it connects straight back to the previous lesson: the cumulative-Gaussian psychometric function is what a signal detection observer traces as the stimulus itself grows stronger. Section 4.11 fits the response curve; this lesson is the decision theory underneath it.

Problem 33 of the practice problems computes d′, c, β and AUC from raw hit and false-alarm counts, including the participant who scored worse on percent correct while being the more sensitive observer.