Statistics · Hypothesis Testing
Type I and Type II Errors
Every statistical test can make two kinds of mistakes. Understanding them — and the trade-off between them — is essential for interpreting research correctly.
When a court finds an innocent person guilty, it has made a terrible mistake. When it acquits a guilty person, it has also made a mistake — of a different kind. Hypothesis testing faces the exact same dilemma. Every test risks two distinct types of error, and reducing one typically increases the other. Understanding this trade-off is fundamental to designing good studies and interpreting their results.
📘 Key Terms
Type I Error (False Positive): Rejecting H₀ when it is actually true. The probability of a Type I error is α (the significance level). In medical testing: concluding a drug works when it doesn’t.
Type II Error (False Negative): Failing to reject H₀ when it is actually false. The probability of a Type II error is β. Power = 1 − β = the probability of correctly rejecting a false H₀.
| H₀ is actually TRUE | H₀ is actually FALSE | |
|---|---|---|
| Reject H₀ | ❌ Type I Error (α) | ✅ Correct (Power = 1−β) |
| Fail to reject H₀ | ✅ Correct (1−α) | ❌ Type II Error (β) |
Real-World Examples
Medical Drug Testing:
Type I: Approving a drug that is actually ineffective (false positive — serious but manageable with monitoring)
Type II: Rejecting a drug that is actually effective (false negative — patients miss a treatment that works)
Type I: Approving a drug that is actually ineffective (false positive — serious but manageable with monitoring)
Type II: Rejecting a drug that is actually effective (false negative — patients miss a treatment that works)
Criminal Justice:
Type I: Convicting an innocent person (false positive — considered the more serious error in law)
Type II: Acquitting a guilty person (false negative — the person goes free)
Type I: Convicting an innocent person (false positive — considered the more serious error in law)
Type II: Acquitting a guilty person (false negative — the person goes free)
💡 Key Insight
Reducing α (e.g. from 0.05 to 0.01) makes it harder to reject H₀, which reduces Type I errors but increases Type II errors. The only way to reduce both simultaneously is to increase sample size, which increases the power of the test and allows more precise estimates of the true effect.
Statistical Power
Power = 1 − β is the probability of correctly detecting a real effect. A power of 0.80 (80%) is commonly considered the minimum acceptable standard in empirical research — it means the test has an 80% chance of detecting a true effect of the assumed size. Factors that increase power: larger sample size, larger true effect size, lower measurement error, higher α.
⚠️ Common Error
Students often confuse α with the probability that H₀ is true, or β with the probability the effect doesn’t exist. Neither is correct. α is the probability of a Type I error given H₀ is true. β is the probability of a Type II error given H₁ is true. Both are conditional probabilities — not unconditional assessments of which hypothesis is correct.
Q1. A drug regulator sets α = 0.01 (rather than 0.05) when approving new drugs. Explain how this affects Type I and Type II errors and discuss the trade-off involved. [6 marks]
Answer: Setting α = 0.01 reduces the probability of a Type I error (approving an ineffective drug) from 5% to 1% — a stricter criterion. However, this increases the probability of a Type II error (β): with a stricter threshold, the test is less likely to detect a truly effective drug, meaning some beneficial drugs may not receive approval. The trade-off depends on the relative costs: in pharmaceutical regulation, Type I errors (approving harmful or ineffective drugs) are considered very costly in terms of public health risk, justifying a low α despite the increased risk of Type II errors. The optimal α depends on the relative costs of each error type.
References
1. Moore, D.S., McCabe, G.P. and Craig, B.A. (2021) Introduction to the Practice of Statistics. W.H. Freeman.
2. Cohen, J. (1988) Statistical Power Analysis for the Behavioral Sciences. 2nd edn. Lawrence Erlbaum.
3. Field, A. (2018) Discovering Statistics Using IBM SPSS Statistics. SAGE.
4. Triola, M.F. (2022) Elementary Statistics. Pearson.
5. Neyman, J. and Pearson, E.S. (1933) ‘On the Problem of the Most Efficient Tests of Statistical Hypotheses’, Philosophical Transactions of the Royal Society A, 231, pp. 289–337.