Step 10: Sampling Distributions and the Central Limit Theorem
- A parameter (such as \( \mu \) or \( p \)) describes the whole population and is usually unknown. A statistic (such as \( \bar{x} \) or \( \hat{p} \)) comes from a sample and changes from sample to sample.
- The sampling distribution of \( \bar{x} \) is centred on \( \mu \) and has spread \( SE = \sigma/\sqrt{n} \), called the standard error. Quadrupling \( n \) halves the standard error.
- The Central Limit Theorem: for a large enough sample (usually \( n \ge 30 \)), \( \bar{x} \) is roughly normal, whatever the shape of the population.
- Exam point: for a sample mean, use \( z = (\bar{x} – \mu) \div (\sigma/\sqrt{n}) \). Dividing by \( \sigma \) alone is the most common mistake.
Parameters and statistics
A poll asks 1,000 people a question and reports that 43% agree, “accurate to within about 3 percentage points”. The country has 50 million people, so the poll asked only 0.002% of them. How can such a tiny group speak for everyone? And how can anyone put a number on how close it is? This step answers both questions. It is the bridge from probability to the rest of statistics.
Start with two words that sound alike but mean different things.
| Quantity | Parameter (population) | Statistic (sample) |
|---|---|---|
| Mean | \( \mu \) (“mu”) | \( \bar{x} \) (“x-bar”) |
| Standard deviation | \( \sigma \) (“sigma”) | \( s \) |
| Proportion | \( p \) | \( \hat{p} \) (“p-hat”) |
| Size | \( N \) | \( n \) |
A university has 20,000 students. A researcher surveys 400 and finds they study 14.2 hours a week on average, and 38% have a part-time job. Here \( N = 20{,}000 \), \( n = 400 \), \( \bar{x} = 14.2 \) and \( \hat{p} = 0.38 \). The true mean \( \mu \) and true proportion \( p \) for all 20,000 students are the parameters. They are what the researcher really wants to know. All of statistical inference comes down to this: use a statistic to learn about a parameter.
Sampling variability and the sampling distribution
Take a random sample and work out its mean. Take another sample and do it again. The two means will differ. Nothing went wrong; different people simply ended up in each sample. This is sampling variability, and you cannot avoid it.
The clever move is to model that variation instead of worrying about it. Imagine taking every possible sample of size \( n \), working out the mean of each, and putting all those means into one distribution.
Like any distribution, it has a centre, a spread and a shape. Two facts give the first two, whatever the population looks like:
- Its centre is \( \mu \). Sample means miss in both directions but do not lean one way. A statistic like this is called unbiased.
- Its spread is \( \sigma/\sqrt{n} \), called the standard error of the mean.
The \( \sqrt{n} \) matters a lot. As \( n \) grows, the standard error shrinks, so sample means cluster more tightly around \( \mu \). But because it is a square root, doubling your accuracy always takes four times as much data.
The Central Limit Theorem
We now know the centre and spread of the sampling distribution. What about its shape? If the population is normal, \( \bar{x} \) is normal for any \( n \). The surprise is what happens when the population is not normal.
Think of service times at a call centre. Most calls are short, a few are very long, so the population is strongly right-skewed. With samples of 2, the means are still skewed. With samples of 10, they are much more even. By samples of 30, a histogram of the means is hard to tell apart from a bell curve. The population did not change. Only the sample size did.
This also explains why the bell curve from Step 8 appears so often. Many real quantities, such as height, are the result of lots of small independent influences added together, and the CLT says such sums tend towards normal.
Using the CLT to find probabilities
All of Step 8’s z-score tools now work for \( \bar{x} \). The only change is the bottom of the fraction: you divide by the standard error, not by \( \sigma \), because you are dealing with an average of \( n \) values rather than one value.
A population has \( \mu = 70 \) and \( \sigma = 12 \). A random sample of 36 is taken. Find the probability that the sample mean is more than 74.
There is only about a 2.3% chance. For one person, \( z = (74 – 70)/12 \approx 0.33 \), which gives a chance of roughly 37%. Averages vary much less than individuals.
Sample proportions work the same way. Polls report percentages, so we need the sampling distribution of \( \hat{p} \). It is centred on the true \( p \), and it is roughly normal when \( np \ge 10 \) and \( n(1-p) \ge 10 \).
Suppose 20% of households in a city own an electric car, and you sample 64. Check: \( np = 12.8 \) and \( n(1-p) = 51.2 \), both at least 10. The standard error is \( \sqrt{0.2 \times 0.8 / 64} = \sqrt{0.0025} = 0.05 \). The chance that more than 25% of the sample own one is found from \( z = (0.25 – 0.20)/0.05 = 1.00 \), giving \( 1 – 0.8413 = 0.1587 \), about 16%.
Why sample size matters and population size mostly does not
Look at both standard error formulas again. The population size \( N \) is in neither of them. Precision depends on how many people you ask, not on how many you could have asked. A random sample of 1,000 is about as precise for a city of 50,000 as for a country of 50 million. That answers the poll puzzle. The one condition is that the sample is under about 10% of the population, so the observations act as independent. Real samples almost always are.
Think of tasting soup. If the pot is well stirred, one spoonful tells you about the whole pot, whether it holds two litres or two hundred. Stirring is like random sampling. A badly stirred pot cannot be saved by a bigger spoon: a biased sample of 100,000 is worse than a random sample of 1,000, because bias does not shrink as \( n \) grows.
You can also run the standard error formula backwards to plan a study. If you want a standard error no bigger than a target, use \( n = (\sigma \div \text{target SE})^2 \) and always round up. With \( \sigma = 18 \) and a target of 3, \( n = 6^2 = 36 \). Halving the target to 1.5 needs \( n = 12^2 = 144 \), four times as many.
Finally, know what the CLT does not say. It does not make the population normal. It does not make a big pile of raw data normal: 10,000 incomes are still right-skewed. It does not fix a biased sample. And it is not the law of large numbers. That law says a single sample mean gets close to \( \mu \) as \( n \) grows. The CLT describes the shape and spread of all the possible sample means, which is what lets us work out margins of error.
Practise: 6 questions
Try each one on paper first, then check the worked answer. Use the z-table from Step 8.
EasyQ1. A factory has 8,000 workers. A sample of 200 works an average of 41.5 hours a week. Identify \( N \), \( n \) and \( \bar{x} \), and say whether 41.5 is a parameter or a statistic. Show worked answer
41.5 is a statistic, because it was calculated from the sample.
EasyQ2. \( \sigma = 20 \). Find the standard error for \( n = 25 \) and for \( n = 100 \). By what factor did it fall? Show worked answer
\( n = 100 \): \( SE = 20/10 = 2 \).
It halved. Quadrupling \( n \) divides the SE by \( \sqrt{4} = 2 \), not by 4.
MediumQ3. \( \mu = 100 \), \( \sigma = 20 \), \( n = 100 \). Find \( P(\bar{x} < 97) \). Show worked answer
\( z = (97 – 100)/2 = -1.50 \).
Table area to the left: 0.0668, about 6.7%.
MediumQ4. The true proportion is \( p = 0.20 \) and \( n = 64 \). Find \( P(0.15 < \hat{p} < 0.30) \). Show worked answer
\( SE = \sqrt{0.2 \times 0.8/64} = 0.05 \).
Lower: \( z = (0.15 – 0.20)/0.05 = -1.00 \), area 0.1587. Upper: \( z = (0.30 – 0.20)/0.05 = 2.00 \), area 0.9772.
\( P = 0.9772 – 0.1587 = 0.8185 \).
MediumQ5. \( \sigma = 30 \) and a researcher needs a standard error of 5. What sample size is needed? What if the target were 2.5? Show worked answer
For 2.5: \( n = (30 \div 2.5)^2 = 12^2 = 144 \). Halving the target error needs four times the sample.
ExamQ6. A coffee chain’s service times have \( \mu = 210 \) seconds and \( \sigma = 60 \) seconds, and are strongly right-skewed. A manager samples 100 customers. (a) Can she use a normal model for \( \bar{x} \)? Justify. [2] (b) Find the mean and standard error of \( \bar{x} \). [2] (c) Find the probability the sample mean is more than 222 seconds. [2] (d) What sample size would halve the standard error? [1] Show worked answer
(b) Mean of \( \bar{x} \) = 210 s [1]; \( SE = 60/\sqrt{100} = 6 \) s [1].
(c) \( z = (222 – 210)/6 = 2.00 \) [1]; \( P = 1 – 0.9772 = 0.0228 \) [1].
(d) Halving the SE needs four times the sample: \( n = 400 \) [1].
Check yourself
1. What does the Central Limit Theorem say becomes approximately normal as \( n \) grows?
2. A random sample of 1,000 is taken from a city of 50,000 and from a country of 50 million. How do their standard errors compare?
3. You want the probability that the mean of 36 values exceeds 74, with \( \mu = 70 \) and \( \sigma = 12 \). What do you divide by?
Your score: 0 out of 3
The short version
- Parameters (\( \mu, \sigma, p \)) describe the population and are unknown; statistics (\( \bar{x}, s, \hat{p} \)) come from samples and vary. Inference uses statistics to learn about parameters.
- The sampling distribution of \( \bar{x} \) is centred on \( \mu \) with standard error \( \sigma/\sqrt{n} \). Quadrupling \( n \) halves the error.
- The Central Limit Theorem: for large \( n \) (usually 30 or more), \( \bar{x} \) is about normal whatever the population looks like. Use \( z = (\bar{x} – \mu)/(\sigma/\sqrt{n}) \).
- For a proportion, the centre is \( p \) and \( SE = \sqrt{p(1-p)/n} \), provided \( np \ge 10 \) and \( n(1-p) \ge 10 \). Precision depends on \( n \), not \( N \), and no sample size fixes bias.
- The full chapter in Statistics Made Simple has many more worked examples and a full set of tiered practice questions with a complete answer key.
Further reading on EconTutorials
Older in-depth articles on this topic from our blog: