Step 15: Comparing Two Proportions
- To compare two groups on a yes/no question, you estimate \(p_1 – p_2\), the difference between two population proportions, using \(\hat{p}_1 – \hat{p}_2\).
- A two-proportion z-interval is \((\hat{p}_1 – \hat{p}_2) \pm z^* \sqrt{\frac{\hat{p}_1(1-\hat{p}_1)}{n_1} + \frac{\hat{p}_2(1-\hat{p}_2)}{n_2}}\). If 0 is not in the interval, the data suggest a real difference.
- A two-proportion z-test of \(H_0: p_1 = p_2\) uses the pooled proportion \(\hat{p}_c\) in the standard error, because \(H_0\) says the two proportions are equal.
- Exam marks come from the four steps: State, Plan (name the procedure and check conditions for both groups), Do, Conclude in context.
What comparing two proportions means
Suppose two cafés each try a new loyalty card. At café A, 60 out of 100 customers sign up. At café B, 45 out of 100 sign up. Café A’s rate is 15 percentage points higher. Is café A really better at selling the card, or could a gap like this appear just by chance, because of which customers happened to walk in?
That is the question this step answers. You have two separate groups, and for each one you count how many “successes” there are (signed up, passed, voted yes). You then compare the two sample proportions and ask how much of the gap is real and how much could be random variation.
Always define your parameters in words first. For example: “\(p_1\) = the proportion of all customers at café A who would sign up, \(p_2\) = the proportion of all customers at café B who would sign up.” Examiners reward this, and it makes your conclusion much easier to write.
The sampling distribution of \(\hat{p}_1 – \hat{p}_2\)
If you took many pairs of random samples and worked out \(\hat{p}_1 – \hat{p}_2\) each time, the differences would vary. That collection of differences is the sampling distribution. It has three features you need to know.
- Centre: the mean is \(\mu_{\hat{p}_1 – \hat{p}_2} = p_1 – p_2\). The statistic is an unbiased estimator of the true difference.
- Spread: the standard deviation is given below. Notice that the variances add, even though you are subtracting the proportions. Two sources of random error stack up.
- Shape: approximately normal when both samples have at least 10 successes and 10 failures.
This standard deviation formula works when the two samples are independent and each sample is no more than 10% of its population (or the data come from randomly assigning treatments in an experiment).
The two-proportion z-interval
To estimate the true difference, build a confidence interval. You do not know \(p_1\) and \(p_2\), so you replace them with \(\hat{p}_1\) and \(\hat{p}_2\) in the standard deviation. The result is called the standard error.
Use \(z^* = 1.645\) for 90%, \(1.960\) for 95% and \(2.576\) for 99% confidence. For the large counts check here, use the observed numbers of successes and failures in each sample.
For example, a researcher takes a random sample of 240 students aged 16 to 18 and finds that 156 use social media for more than three hours a day. A separate random sample of 260 adults aged 30 to 45 finds 117 who do. Build and interpret a 95% confidence interval for the difference in the two population proportions.
In plain words: the proportion is somewhere between about 11.5 and 28.5 percentage points higher for the students. Because the whole interval is above 0, the data give convincing evidence that the student proportion is higher.
Does zero lie in the interval?
The number 0 is the key value, because \(p_1 – p_2 = 0\) means “no difference”.
- If the interval is entirely above 0, there is convincing evidence that \(p_1 > p_2\).
- If the interval is entirely below 0, there is convincing evidence that \(p_1 < p_2\).
- If the interval contains 0, “no difference” is a plausible value. You do not have convincing evidence of a difference. That does not prove the proportions are equal.
A 95% interval that excludes 0 matches a two-sided test at \(\alpha = 0.05\) that rejects \(H_0: p_1 = p_2\).
The two-proportion z-test
When the question asks “is there convincing evidence of a difference?”, you use a significance test. The null hypothesis is almost always “no difference”:
Here is the twist. A test assumes \(H_0\) is true while calculating, and \(H_0\) says \(p_1 = p_2\). If the two proportions are the same, your best estimate of that one common value combines both samples. This is the pooled (combined) proportion.
Here \(X_1\) and \(X_2\) are the numbers of successes. For the large counts condition in a test, check that \(n_1\hat{p}_c,\ n_1(1-\hat{p}_c),\ n_2\hat{p}_c,\ n_2(1-\hat{p}_c)\) are all at least 10.
For example, a school randomly assigns 232 students to two groups. The 120 students in group 1 use a revision app; 72 of them pass a mock exam. The 112 students in group 2 revise as normal; 56 of them pass. At \(\alpha = 0.05\), is there convincing evidence that the app increases the pass rate?
A 10-point gap sounds big, but with groups of this size a gap that large would happen by chance alone about 6% of the time. That is not rare enough at the 5% level. Note that “fail to reject” does not mean the app is useless; a larger experiment might detect a real effect.
Practise: 6 questions
Try each one on paper first, then check the worked answer.
EasyQ1. In a random sample of 150 adults from town A, 45 cycle to work. In a separate random sample of 90 adults from town B, 36 cycle to work. Find \(\hat{p}_1\), \(\hat{p}_2\) and \(\hat{p}_1 – \hat{p}_2\), and check the large counts condition for an interval. Show worked answer
\(\hat{p}_1 – \hat{p}_2 = 0.30 – 0.40 = -0.10\).
Successes and failures: 45 and 105 in town A; 36 and 54 in town B. All are at least 10, so the condition is met.
EasyQ2. Using the data in Q1, find the pooled proportion \(\hat{p}_c\). Show worked answer
This is the overall proportion who cycle when the two samples are combined. It lies between 0.30 and 0.40, as it must.
MediumQ3. Using the data in Q1, build a 95% confidence interval for \(p_1 – p_2\). Does it give convincing evidence of a difference? Show worked answer
Margin of error \(= 1.96 \times 0.0638 \approx 0.125\).
Interval: \(-0.10 \pm 0.125\), giving \((-0.225,\ 0.025)\).
The interval contains 0, so “no difference” is plausible. There is not convincing evidence that the proportions of adults who cycle to work differ between the two towns.
MediumQ4. A 95% interval for \(p_1 – p_2\) (girls minus boys who play a musical instrument) is \((0.04,\ 0.17)\). Interpret it and say what it suggests. Show worked answer
The whole interval is above 0, so there is convincing evidence that a higher proportion of girls than boys play an instrument: somewhere between 4 and 17 percentage points higher.
MediumQ5. Using the data in Q1, test \(H_0: p_1 = p_2\) against \(H_a: p_1 \ne p_2\). Find the z statistic and p-value. Show worked answer
\(SE = \sqrt{0.3375 \times 0.6625 \left(\dfrac{1}{150} + \dfrac{1}{90}\right)} \approx 0.0630\).
\(z = \dfrac{-0.10}{0.0630} \approx -1.59\).
Two-sided p-value \(= 2 \times P(Z \le -1.59) \approx 0.11\).
Because 0.11 > 0.05, we fail to reject \(H_0\). This matches the interval in Q3, which contained 0.
ExamQ6. In an experiment, 600 volunteers are randomly assigned, 300 to a new vaccine and 300 to a placebo. Over the winter, 18 in the vaccine group and 36 in the placebo group catch the flu. Do the data give convincing evidence at \(\alpha = 0.05\) that the vaccine reduces the proportion who catch the flu? [6 marks] Show worked answer
Plan [2]: Two-proportion z-test. Random: volunteers randomly assigned to treatments. Large counts: \(\hat{p}_c = \dfrac{18 + 36}{600} = 0.09\); \(300(0.09) = 27\) and \(300(0.91) = 273\) for each group, all at least 10.
Do [2]: \(\hat{p}_V = 18/300 = 0.06\), \(\hat{p}_P = 36/300 = 0.12\).
\(SE = \sqrt{0.09 \times 0.91 \left(\dfrac{1}{300} + \dfrac{1}{300}\right)} \approx 0.0234\).
\(z = \dfrac{0.06 – 0.12}{0.0234} \approx -2.57\), p-value \(= P(Z \le -2.57) \approx 0.005\).
Conclude [1]: Because the p-value of 0.005 is less than \(\alpha = 0.05\), we reject \(H_0\). We have convincing evidence that the vaccine reduces the true proportion of volunteers like these who catch flu. Because treatments were randomly assigned, we can conclude the vaccine caused the reduction.
Check yourself
1. In a two-proportion z-test of \(H_0: p_1 = p_2\), the standard error uses:
2. A 95% interval for \(p_1 – p_2\) is \((-0.03,\ 0.08)\). What can you conclude?
3. The standard deviation of \(\hat{p}_1 – \hat{p}_2\) is found by:
Your score: 0 out of 3
The short version
- \(\hat{p}_1 – \hat{p}_2\) has mean \(p_1 – p_2\) and standard deviation \(\sqrt{\frac{p_1(1-p_1)}{n_1} + \frac{p_2(1-p_2)}{n_2}}\); it is roughly normal with large counts in both groups.
- Conditions: Random (samples or assignment), 10% for each sample, at least 10 successes and 10 failures in each group.
- Interval: \((\hat{p}_1 – \hat{p}_2) \pm z^* \times\) unpooled SE. If 0 is outside, there is convincing evidence of a difference.
- Test of \(H_0: p_1 = p_2\): use the pooled \(\hat{p}_c = \frac{X_1 + X_2}{n_1 + n_2}\) in the standard error.
- Always define both parameters, follow State–Plan–Do–Conclude, and conclude in context without “accepting” \(H_0\).
Further reading on EconTutorials
Older in-depth articles on this topic from our blog: