Statistics · Inference
Type I vs Type II Errors: A Way to Never Confuse Them Again
A false positive and a false negative. Everyone mixes them up. Here’s the mnemonic, the trade-off nobody explains, and why one error is usually worse than the other.
Every hypothesis test can be wrong in exactly two ways. You can cry wolf when there’s no wolf, or you can miss the wolf that’s really there.
Statisticians named these Type I and Type II — possibly the least helpful names in the entire field, because the numbers tell you nothing about which is which. This post fixes that, gives you a mnemonic that sticks, and then does the part most courses skip: explains why the two errors trade off against each other, and how to decide which one you’d rather risk.
The two errors
| Null is really TRUE | Null is really FALSE | |
|---|---|---|
| You reject the null | ❌ Type I error (false positive) | ✅ Correct |
| You fail to reject | ✅ Correct | ❌ Type II error (false negative) |
🔑 Key terms
Type I error (α) — rejecting a true null. You claimed an effect that isn’t there. A false alarm.
Type II error (β) — failing to reject a false null. You missed an effect that is real. A miss.
💡 Insight — the mnemonic that sticks
“The boy who cried wolf” tells the whole story in order.
First the boy cries wolf when there’s no wolf — a false alarm. That’s the first error, Type I.
Then a real wolf comes and nobody believes him — a real threat missed. That’s the second error, Type II. The fable literally runs Type I then Type II. You’ll never flip them again.
Another anchor: Type I is the one you control directly. When you set your significance level α to 0.05, you are choosing to accept a 5% chance of a Type I error. That’s the deal you strike before you ever see the data.
The trade-off nobody explains
Here’s the part that’s usually missing. The two errors are in tension: lowering one raises the other.
If you demand overwhelming evidence before declaring an effect (a tiny α, say 0.001), you almost never cry wolf — Type I errors become rare. But that same caution means you’ll miss real, modest effects — Type II errors go up. Set the bar high enough and you’ll reject almost nothing, true or false.
Turn it around: make α generous (0.10) and you’ll catch more real effects, but you’ll also raise more false alarms. You can’t drive both to zero at once with a fixed sample. It’s a see-saw.
💡 Insight — the only way to reduce both at once
There’s exactly one escape from the see-saw: collect more data. A bigger sample sharpens the whole test, letting you lower the false-negative rate without raising the false-positive rate. This is what “statistical power” is really about — and it’s why a well-designed study decides its sample size before collecting anything, by asking how small an effect it needs to be able to catch.
🔑 Key term
Power = 1 − β — the probability of correctly detecting an effect that is really there. High power means low Type II risk. You raise power with a bigger sample, a larger true effect, less noise, or a more lenient α.
📊 Case study — which error would you rather make?
The “right” balance depends entirely on the stakes, and this is a decision, not a calculation.
A smoke alarm should be tuned to almost never miss a fire (low Type II) even at the cost of occasional false alarms from burnt toast (higher Type I). A missed fire is catastrophic; a false alarm is annoying. You’d rather cry wolf.
A criminal trial is the opposite. “Beyond reasonable doubt” is a deliberately tiny α: the system would rather let guilty people go free (Type II) than convict the innocent (Type I). Blackstone’s ratio — “better ten guilty escape than one innocent suffer” — is literally a statement about preferring Type II errors to Type I.
A cancer screening test leans toward catching every real case (low Type II), accepting false positives that a follow-up test will sort out. The choice of which error to fear is a values question wearing a statistics costume — and pretending it’s purely technical is how bad policy gets made.
Practice questions
Q1. A drug trial concludes a useless drug works. Which error is that?
Q2. A trial concludes an effective drug doesn’t work. Which error?
Q3. A researcher lowers α from 0.05 to 0.01 to “be more rigorous.” What happens to the Type I rate? To the Type II rate? To power?
Q4. You’re designing a test to detect contamination in drinking water. Which error should you fear more, and how would you tune the test?
Worked answers
A1. Type I. The null (“drug has no effect”) is true, but you rejected it. A false positive — you cried wolf.
A2. Type II. The null is false (the drug does work), but you failed to reject it. A false negative — you missed the wolf.
A3. Lowering α to 0.01 reduces the Type I rate (fewer false positives — that was the goal). But it raises the Type II rate, because you now demand stronger evidence and will miss more real effects. And since power = 1 − β, power falls. “More rigorous” against one error is “less sensitive” to the other — there’s no free lunch without more data.
A4. Fear the Type II error — declaring safe water contaminated (Type I) means an unnecessary boil notice; declaring contaminated water safe (Type II) makes people ill. Tune the test toward high sensitivity: a more generous α, and crucially enough sampling to give high power against even low contamination levels. As with the smoke alarm, you’d rather have false alarms than misses.
The short version
• Type I = false positive = crying wolf = rejecting a true null. You control it with α.
• Type II = false negative = missing the wolf = failing to reject a false null. Its rate is β.
• “Cried wolf” runs Type I then Type II, in order.
• The two trade off — lowering one raises the other, unless you collect more data.
• Power = 1 − β. Which error to fear is a values judgement, not a formula.
• Type II = false negative = missing the wolf = failing to reject a false null. Its rate is β.
• “Cried wolf” runs Type I then Type II, in order.
• The two trade off — lowering one raises the other, unless you collect more data.
• Power = 1 − β. Which error to fear is a values judgement, not a formula.
This is the flip side of what a p-value means — α is just the Type I rate you agreed to in advance.
References
1. Neyman, J. & Pearson, E.S. (1933) “On the Problem of the Most Efficient Tests of Statistical Hypotheses,” Philosophical Transactions of the Royal Society A, 231, pp. 289–337.
2. Cohen, J. (1988) Statistical Power Analysis for the Behavioral Sciences. 2nd edn. Hillsdale, NJ: Erlbaum.
Power and error rates get a full chapter in Statistics Made Simple.
Including why most published studies are underpowered — and what that quietly does to the results you cite.