Statistics · Probability Theory
The Central Limit Theorem
Why sample means always tend toward a normal distribution — regardless of the shape of the original population. The most remarkable result in all of statistics.
Suppose you’re studying the income of taxi drivers in a city. Incomes are heavily skewed — most earn around £25,000 but a few top earners make £80,000+. Now suppose you take random samples of 30 drivers and calculate the mean income in each sample. Repeat this 1,000 times. Remarkably, the distribution of those 1,000 sample means will be approximately normal — even though the underlying incomes are highly skewed. This is the Central Limit Theorem (CLT).
📘 Key Term
The Central Limit Theorem states that, for sufficiently large samples drawn from any population with a finite mean (μ) and variance (σ²), the sampling distribution of the sample mean (x̄) will be approximately normally distributed with mean μ and standard error σ/√n — regardless of the shape of the original population distribution.
The Formal Statement
If X₁, X₂, …, Xₙ are i.i.d. with mean μ and variance σ², then:
x̄ ~ N(μ, σ²/n) approximately, for large n
Standard Error of the Mean (SEM) = σ/√n
What ‘Large Enough’ Means
| Population Shape | Sample Size Required |
|---|---|
| Approximately normal already | n ≥ 10 often sufficient |
| Moderate skew | n ≥ 30 (the common rule of thumb) |
| Heavy skew or outliers | n ≥ 50 or more |
💡 Key Insight
The CLT is why sample means are normally distributed even when individual data points aren’t. This underpins virtually every statistical test — t-tests, ANOVA, regression — because those tests rely on the normality of the sampling distribution, not the individual data. The CLT means these tests are robust to non-normal populations when sample sizes are large.
Worked Example: Manufacturing Quality Control
A factory produces bolts with population mean diameter μ = 10mm and σ = 0.5mm (distribution unknown).
A quality inspector takes a sample of n = 36 bolts.
By the CLT: x̄ ~ N(10, 0.5²/36) = N(10, 0.0069) approximately
Standard Error = 0.5/√36 = 0.5/6 = 0.0833mm
P(x̄ > 10.2) = P(Z > (10.2−10)/0.0833) = P(Z > 2.40) = 1−0.9918 = 0.82%
⚠️ Common Error
Students sometimes apply the CLT to individual observations rather than sample means. The CLT applies to the distribution of sample means — not to individual data points. A single observation drawn from a skewed population is still drawn from a skewed distribution; it is only the mean of many observations that becomes approximately normal.
Q1. A population of household debts is heavily right-skewed with μ = £8,500 and σ = £3,200. A random sample of n = 64 households is taken. Describe the sampling distribution of x̄ and calculate P(x̄ > £9,000). [6 marks]
Answer: By the CLT, x̄ is approximately normally distributed: x̄ ~ N(8500, 3200²/64) = N(8500, 160000). Standard Error = 3200/√64 = 3200/8 = 400. Z = (9000−8500)/400 = 500/400 = 1.25. P(Z > 1.25) = 1 − 0.8944 = 0.1056 = 10.56%. Despite the heavy skew of the population, with n = 64 the CLT ensures the sampling distribution is approximately normal.
References
1. Moore, D.S., McCabe, G.P. and Craig, B.A. (2021) Introduction to the Practice of Statistics. W.H. Freeman.
2. Hogg, R.V., McKean, J. and Craig, A.T. (2019) Introduction to Mathematical Statistics. Pearson.
3. Rice, J.A. (2007) Mathematical Statistics and Data Analysis. 3rd edn. Cengage.
4. Freedman, D., Pisani, R. and Purves, R. (2007) Statistics. W.W. Norton.
5. Billingsley, P. (1995) Probability and Measure. Wiley.