Statistical Power Explained: What Affects Power and How to Choose a Sample Size

In 30 seconds

Statistical power is the probability that a significance test rejects $H_0$ when a specific alternative is true. Power $= 1 – \beta$, where $\beta$ is the chance of a Type II error.
Power rises when the sample is larger, the true effect is bigger, the data are less variable, or the significance level $\alpha$ is higher.
A common planning target is power 0.80, and the sample size for a z test is $n = \left((z_\alpha + z_\beta)\sigma/\delta\right)^2$, rounded up.
The case answer: a 25-student trial of a tutoring scheme has only a 64% chance of detecting a real 6-point gain; 39 students lift that to 80%.
Exam point: power is always calculated for a particular alternative value. State which one you used, and describe power in context.

The case: is the tutoring trial big enough?

A school runs a statistics exam each year. Over many years the mean mark has been 60 out of 100, with a standard deviation of 15. (The figures are illustrative.) The head of maths wants to trial a new after-school tutoring scheme. She believes it could lift the mean by about 6 marks, which would be worth the cost. If it does less than that, it is not worth running.

She has funding for 25 students in the trial. At the end, she will test $H_0: \mu = 60$ against $H_a: \mu > 60$ at the 5% significance level, treating the 25 as a random sample of students who would join the scheme.

Her worry is simple. Suppose the scheme really does work and raises the mean to 66. How likely is it that her trial will actually show a significant result? If that chance is low, the trial might wrongly suggest the scheme is useless, and a good idea would be dropped. That chance is the test’s power, and it is what she should check before spending any money.

The concept: statistical power

Every significance test can go wrong in two ways. A Type I error rejects a true null hypothesis; its probability is the significance level $\alpha$. A Type II error fails to reject a false null hypothesis; its probability is called $\beta$. Power is the flip side of a Type II error: the probability of correctly rejecting $H_0$ when it is false.^[1]

There is a catch. “$H_0$ is false” covers many possibilities: the true mean could be 61, 66 or 75. A tiny improvement is hard to detect, and a big one is easy. So power is not one number. It depends on which alternative value is true, and we always calculate it for a specific value, usually the smallest effect that would matter in practice.

The calculation has two stages. First, find the rejection region assuming $H_0$ is true. Second, find the probability of landing in that region assuming the alternative is true. Power links the two sampling distributions: the one under $H_0$ and the one under $H_a$.

Key termStatistical power: the probability that a significance test, at a given level $\alpha$, rejects the null hypothesis when a particular alternative value of the parameter is true. Power $= 1 – \beta = P(\text{reject } H_0 \mid H_a \text{ true})$.

\[ \text{Power} = P\!\left(Z \ge z_\alpha – \frac{\delta}{\sigma/\sqrt{n}}\right), \qquad n = \left(\frac{(z_\alpha + z_\beta)\,\sigma}{\delta}\right)^{2} \]

These formulas are for a one-sided z test of a mean when $\sigma$ is known. $\delta$ is the size of the effect we want to detect: the true mean minus the null value (here $66 – 60 = 6$). $\sigma$ is the population standard deviation and $n$ the sample size, so $\sigma/\sqrt{n}$ is the standard error of the sample mean. $z_\alpha$ is the critical value for the test (1.645 for $\alpha = 0.05$, one-sided). $z_\beta$ is the z-value with probability $\beta$ above it (0.842 for power 0.80; 1.282 for power 0.90). For a two-sided test, replace $z_\alpha$ with $z_{\alpha/2}$ (1.96 at the 5% level); this ignores the tiny chance of rejecting in the wrong tail.

Statistics Made Simple explains statistical power and Type II errors with worked examples, diagrams and tables.Get the textbook · $12

Why this method fits the case

The head of maths has a clear null value (60), a clear direction (she only cares about improvement), a known spread from years of results (15), and a smallest effect worth finding (6 marks). Those are exactly the ingredients a power calculation needs. Because she is still planning, power analysis lets her change the design before collecting data, which is when it is most useful.

Two simplifications are worth stating. We treat $\sigma = 15$ as known, which is reasonable with many years of data; in practice a t test would be used, and its power is slightly lower for small samples. We also assume the 25 trial students behave like a random sample of future participants.

When is a power calculation not the right tool?

After the study, to explain a non-significant result. “Observed power” computed from the sample’s own effect adds nothing beyond the P-value. Power belongs at the design stage, or for an effect size chosen in advance.
When you have no idea of the effect size or the spread. A power calculation built on guesses gives a false sense of precision. Run a small pilot study first.
When the design is not a simple one-sample test. Comparisons of groups, paired designs and regressions have their own power formulas, usually handled with software.

Setting it up

Quantity	Symbol	Value
Null hypothesis mean	$\mu_0$	60
Alternative mean of interest	$\mu_a$	66
Effect size	$\delta$	6
Population standard deviation	$\sigma$	15
Sample size	$n$	25
Standard error	$\sigma/\sqrt{n}$	$15/5 = 3$
Significance level (one-sided)	$\alpha$	0.05, $z_\alpha = 1.645$

It also helps to describe the effect in standard deviation units: $\delta/\sigma = 6/15 = 0.4$. An effect of 0.4 SD is often described as small-to-medium, and effects of this size need moderate samples to detect reliably.^[2]

Solving it

Worked example

With $n = 25$, $\sigma = 15$, $\alpha = 0.05$ (one-sided), find the power of the test of $H_0: \mu = 60$ when the true mean is 66, and the probability of a Type II error.

Step 1Rejection region under $H_0$: reject if $\bar{x} \ge 60 + 1.645 \times 3 = 64.93$.

Step 2Now assume $\mu = 66$. The sample mean is $N(66, 3^2)$.

Step 3Standardise the cut-off: $z = (64.93 – 66)/3 \approx -0.355$.

Step 4Power $= P(Z \ge -0.355) \approx 0.639$. Then $\beta = 1 – 0.639 = 0.361$.

If the scheme really raises the mean to 66, a 25-student trial detects it only about 64% of the time. More than a third of the time it would miss a real effect.

A 36% chance of missing a real improvement is uncomfortable. The second example finds how many students would give the conventional power of 0.80.

Worked example

Find the sample size needed for power 0.80 against $\mu = 66$, with $\alpha = 0.05$ one-sided. Repeat for power 0.90, and for a two-sided test with power 0.80.

Step 1Power 0.80: $n = \left((1.645 + 0.842) \times 15/6\right)^2 = (6.217)^2 \approx 38.6$, so $n = 39$.

Step 2Check: SE $= 15/\sqrt{39} \approx 2.402$; power $= P(Z \ge 1.645 – 6/2.402) = P(Z \ge -0.853) \approx 0.803$.

Step 3Power 0.90: $n = \left((1.645 + 1.282) \times 2.5\right)^2 \approx 53.5$, so $n = 54$.

Step 4Two-sided, power 0.80: $n = \left((1.96 + 0.842) \times 2.5\right)^2 \approx 49.1$, so $n = 50$.

The trial needs 39 students, not 25, to have an 80% chance of detecting a 6-mark gain.

The third example changes one ingredient at a time to show what drives power.

Worked example

Starting from $n = 25$, $\sigma = 15$, $\alpha = 0.05$ one-sided and $\mu_a = 66$ (power 0.639), find the power when (a) the true gain is 3 marks, (b) the true gain is 9 marks, (c) $\alpha = 0.01$, (d) $n = 100$.

Step 1(a) $\delta = 3$: power $= P(Z \ge 1.645 – 1) = P(Z \ge 0.645) \approx 0.260$.

Step 2(b) $\delta = 9$: power $= P(Z \ge 1.645 – 3) = P(Z \ge -1.355) \approx 0.912$.

Step 3(c) $\alpha = 0.01$, $z_\alpha = 2.326$: cut-off 66.98, power $= P(Z \ge 0.326) \approx 0.372$.

Step 4(d) $n = 100$, SE = 1.5: power $= P(Z \ge 1.645 – 4) = P(Z \ge -2.355) \approx 0.991$.

Bigger effects and bigger samples raise power sharply; a stricter $\alpha$ lowers it.

Make power and Type II error calculations automatic with the Practice Questions book: 1,569 questions, all fully worked.Get 1,569 questions · $9

What the answer tells us

As planned, the trial is underpowered. Even if the scheme works exactly as hoped, there is a 36% chance the results will not be significant, and the school may wrongly conclude that tutoring does not help. Increasing the trial to 39 students cuts that risk to 20%. The extra 14 places cost money, but much less than throwing away a scheme that works, or running a second trial later.

Reasonableness check. With $n = 25$ the cut-off is 64.93, only about a third of a standard error (1.07 marks out of 3) below 66. If the true mean is 66, the sample mean lands above the centre half the time and between 64.93 and 66 a little more often, so the power should be a bit above one half. A power of about 0.64 fits that picture. Also, if the true mean were exactly 60, the power formula gives $P(Z \ge 1.645) = 0.05$, which is $\alpha$. Power always starts at $\alpha$ when there is no effect.

Sensitivity. Figure 1 shows power curves for 25 and 39 students. Each curve starts at 0.05 when there is no improvement and rises towards 1 as the true improvement grows. At a 6-mark gain, the 25-student design sits at 0.64, below the dashed 0.80 line; the 39-student design reaches it. Notice how flat the curves are for small effects: a 3-mark gain would be missed about three times out of four with 25 students. If the head of maths cared about gains that small, she would need about 155 students for 80% power.

Power of the one-sided test, α = 0.05, σ = 15 probability of rejecting H₀

n = 25n = 39Target power 0.80

Figure 1. Power rises with the true size of the improvement and with the sample size. At a 6-mark gain, 25 students give power 0.64 and 39 students give 0.80. Illustrative figures.

The four levers can be summarised in one sentence: power goes up with a larger $n$, a larger true effect $\delta$, a smaller spread $\sigma$, and a larger $\alpha$. Only $n$ and $\alpha$ are usually under the researcher’s control, and raising $\alpha$ means accepting more false alarms. Reducing $\sigma$ is sometimes possible through better measurement or a paired design, which is why careful study design can be as valuable as a bigger sample.^[3]

Common mistakeCalculating the rejection region with the alternative mean. The cut-off (64.93 here) always comes from the null distribution, centred at 60. Only after you have it do you switch to the alternative distribution, centred at 66, to find the probability of landing beyond it.

Common mistakeSaying “the power of the test is 0.64” without naming the alternative. Power changes with the true value: the same test has power 0.26 against a 3-mark gain and 0.91 against a 9-mark gain. Always say “power against $\mu = 66$” or describe it in context.

Exam tipExam questions on power often ask you to interpret it or to say how it would change, rather than to calculate it. A model interpretation: “If the true mean mark is 66, there is a 0.64 probability that the test will correctly reject $H_0$ and conclude the scheme raises marks.” Power and Type II errors appear in AP Statistics, and the Cambridge AS & A Level Mathematics syllabus includes calculating probabilities of Type I and Type II errors in its later statistics content.^[4]^[5] Our statistics guide shows where power fits after the foundations of hypothesis testing.

The Statistics Made Simple textbook explains power with paired diagrams of the null and alternative distributions, which make the two-stage calculation easy to picture. The Statistics Made Simple Practice Questions book includes graded Type II error and power questions with full solutions.

Master significance tests, errors and power with both Statistics Made Simple books for $16.Both books · $16

Practice questions

Use $z_{0.05} = 1.645$, $z_{0.025} = 1.96$, $z_{0.20} = 0.842$ and $z_{0.10} = 1.282$ where needed.

EasyQ1. For a fixed alternative, state whether power increases or decreases when (a) $n$ increases, (b) $\alpha$ is changed from 0.05 to 0.01, (c) the population SD is larger. Show worked answer

(a) Increases: the standard error shrinks, so the two distributions overlap less.
(b) Decreases: the rejection region gets smaller.
(c) Decreases: a larger SD widens both distributions and increases their overlap.

EasyQ2. A test has power 0.85 against a particular alternative. Find $\beta$ and interpret both numbers. Show worked answer

$\beta = 1 – 0.85 = 0.15$.
If that alternative is true, the test rejects $H_0$ with probability 0.85 and fails to reject (a Type II error) with probability 0.15.

MediumQ3. Test $H_0: \mu = 50$ against $H_a: \mu > 50$ with $\sigma = 12$, $n = 36$, $\alpha = 0.05$. Find the power when $\mu = 55$. Show worked answer

SE $= 12/6 = 2$. Reject if $\bar{x} \ge 50 + 1.645 \times 2 = 53.29$.
If $\mu = 55$: $z = (53.29 – 55)/2 \approx -0.855$.
Power $= P(Z \ge -0.855) \approx 0.804$.

MediumQ4. A one-sided test at $\alpha = 0.05$ is planned with $\sigma = 10$. What sample size gives power 0.80 for detecting an increase of 4 units in the mean? Show worked answer

$n = \left((1.645 + 0.842) \times 10/4\right)^2 = (6.2175)^2 \approx 38.66$.
Round up: $n = 39$.

ExamQ5. A machine fills bags with mean 100 g and SD 10 g. After a service, a manager tests $H_0: \mu = 100$ against $H_1: \mu > 100$ at the 5% level using a random sample of 16 bags. (a) Find the critical value of $\bar{x}$. [2] (b) Find the power of the test if the true mean is 104 g. [3] (c) State the probability of a Type II error in this case. [1] (d) Suggest one way to increase the power. [1] Show worked answer

(a) SE $= 10/4 = 2.5$ [M1]; critical value $100 + 1.645 \times 2.5 = 104.11$ g [A1].
(b) If $\mu = 104$: $P(\bar{x} \ge 104.11)$ [M1] $= P(Z \ge 0.045)$ [M1] $\approx 0.482$ [A1].
(c) $\beta = 1 – 0.482 = 0.518$ [B1].
(d) Take a larger sample (or use a higher significance level) [B1].

Frequently asked questions

Why is 0.80 the usual target for power?

It is a convention popularised by the psychologist Jacob Cohen, who suggested that a Type II error rate of 0.20 was a reasonable balance against a Type I error rate of 0.05.^[2] It is not a law. Medical trials, where missing a real effect can be costly, often aim for 0.90.

How are power and Type II errors related?

They are complements for the same alternative: power $= 1 – \beta$. A test with power 0.64 has a 0.36 chance of a Type II error against that alternative. Our post on Type I and Type II errors covers both error types and their trade-off.

Can I increase power without collecting more data?

Sometimes. A one-sided test has more power than a two-sided test if the direction is truly known in advance. Better measurement reduces $\sigma$. A paired or matched design removes variation between subjects. Raising $\alpha$ also raises power, but at the cost of more false positives, so it is rarely the right choice.

Does a non-significant result mean there is no effect?

No. A low-powered study can easily miss a real effect. “Not significant” means the data did not provide convincing evidence, not that the null hypothesis is true. That is why reporting power, or a confidence interval for the effect, alongside the P-value helps readers judge a null result. See our P-value guide for more.

References

Illowsky, B. and Dean, S. (2023) Introductory Statistics 2e. Houston, TX: OpenStax. https://openstax.org/details/books/introductory-statistics-2e
Cohen, J. (1988) Statistical Power Analysis for the Behavioral Sciences. 2nd edn. Hillsdale, NJ: Lawrence Erlbaum Associates.
Moore, D. S., McCabe, G. P. and Craig, B. A. Introduction to the Practice of Statistics. New York: W. H. Freeman.
College Board. AP Statistics Course and Exam Description. New York: College Board.
Cambridge International. Cambridge International AS & A Level Mathematics 9709: Syllabus for examination in 2026 and 2027. https://www.cambridgeinternational.org/programmes-and-qualifications/cambridge-international-as-and-a-level-mathematics-9709/

Statistical Power Explained: What Affects Power and How to Choose a Sample Size

The case: is the tutoring trial big enough?

The concept: statistical power

Why this method fits the case

Setting it up

Solving it

What the answer tells us

Practice questions

Frequently asked questions

Why is 0.80 the usual target for power?

How are power and Type II errors related?

Can I increase power without collecting more data?

Does a non-significant result mean there is no effect?

References

Keep reading

Contact Info

Quick Links

Success!