Suppose you want to test whether four different fertilizers produce different average crop yields. Your first instinct might be to run six separate t-tests — comparing fertilizer 1 vs 2, 1 vs 3, 1 vs 4, 2 vs 3, 2 vs 4, and 3 vs 4. This is a mistake, and not a small one. Each t-test carries a 5% chance of a false positive at α = 0.05; running six of them pushes your overall chance of at least one false positive well above 5% — a problem called the familywise error rate. Analysis of Variance (ANOVA) solves this by testing all groups simultaneously with a single test.
What Is ANOVA?
One-way ANOVA tests whether the means of three or more groups are equal, using a single categorical factor (in this example, “fertilizer type”) to define the groups. Despite the name “Analysis of Variance,” ANOVA is fundamentally a test about means — it just uses variance calculations as the mechanism to compare them, which is where the apparent contradiction in the name comes from.
The hypotheses are straightforward:
H₀: μ₁ = μ₂ = μ₃ = … = μk (all group means are equal)
H₁: at least one group mean is different
Notice that H₁ does not say all means differ — only that at least one does. A significant ANOVA result tells you a difference exists somewhere among the groups, but not which specific groups differ; that requires a follow-up post-hoc test, covered below.
The Logic: Between-Group vs Within-Group Variance
ANOVA works by splitting the total variability in the data into two sources. Between-group variance measures how much the group means differ from the overall (“grand”) mean — large if the groups are genuinely different. Within-group variance measures how much individual observations vary around their own group’s mean — this reflects natural random noise unrelated to the grouping factor.
The test statistic, called F, is the ratio of these two:
F = Between-group variance / Within-group variance
If the null hypothesis is true and all group means are genuinely equal, F should be close to 1 — between-group differences are no larger than you’d expect from random noise alone. If the group means genuinely differ, between-group variance will be inflated relative to within-group variance, pushing F well above 1.
Worked Example: Four Fertilizers
Suppose 4 plots are tested with each of 3 fertilizers (n = 3 per group, k = 3 groups for a simpler illustration), giving these crop yields (kg):
| Fertilizer A | Fertilizer B | Fertilizer C |
|---|---|---|
| 20, 22, 24 | 28, 30, 32 | 18, 20, 19 |
Group means: Ā = 22, B̄ = 30, C̄ = 19. Grand mean (all 9 values) = (22+30+19)/3 = 23.67.
Step 1: Sum of Squares Between (SSB) — how far each group mean is from the grand mean, weighted by group size:
SSB = 3(22−23.67)² + 3(30−23.67)² + 3(19−23.67)²
= 3(2.79) + 3(40.09) + 3(21.81) = 8.36 + 120.27 + 65.42 = 194.05
Step 2: Sum of Squares Within (SSW) — how far each individual value is from its own group’s mean:
Group A: (20−22)²+(22−22)²+(24−22)² = 4+0+4 = 8
Group B: (28−30)²+(30−30)²+(32−30)² = 4+0+4 = 8
Group C: (18−19)²+(20−19)²+(19−19)² = 1+1+0 = 2
SSW = 8+8+2 = 18
Step 3: Degrees of freedom and mean squares. df between = k−1 = 3−1 = 2. df within = N−k = 9−3 = 6.
MSB = SSB / dfbetween = 194.05 / 2 = 97.03
MSW = SSW / dfwithin = 18 / 6 = 3.00
Step 4: The F-statistic.
F = MSB / MSW = 97.03 / 3.00 ≈ 32.3
The ANOVA Table
| Source | SS | df | MS | F |
|---|---|---|---|---|
| Between | 194.05 | 2 | 97.03 | 32.3 |
| Within | 18.00 | 6 | 3.00 | — |
| Total | 212.05 | 8 | — | — |
The critical value for F with dfbetween=2 and dfwithin=6 at α=0.05 is approximately 5.14. Since 32.3 > 5.14, we reject H₀ — there is strong evidence that at least one fertilizer produces a different mean yield than the others.
Which Groups Actually Differ? Post-Hoc Tests
A significant ANOVA result only tells you that somewhere among the groups, a difference exists — it does not identify which pairs of groups differ. To find that out, you run a post-hoc test after a significant ANOVA, such as Tukey’s HSD (Honestly Significant Difference) test, which compares every pair of groups while controlling the familywise error rate — exactly the problem that running raw pairwise t-tests would have created in the first place. This two-step approach (ANOVA first, post-hoc only if significant) is standard practice and prevents the error-rate inflation described at the start of this post.
Assumptions of ANOVA
Independence: observations within and across groups must be independent of each other. Normality: the data within each group should be approximately normally distributed (ANOVA is fairly robust to mild violations, especially with larger sample sizes). Homogeneity of variance: the groups should have roughly equal variances — formally tested with Levene’s test. When this assumption fails badly, an alternative like Welch’s ANOVA, which doesn’t assume equal variances, is more appropriate.
Common Mistakes
Running multiple t-tests instead of ANOVA. As shown at the start, this inflates the familywise error rate and is the single most common reason to reach for ANOVA in the first place.
Interpreting a significant F as meaning all groups differ. A significant ANOVA only establishes that at least one group differs from at least one other — identifying which requires a post-hoc test.
Skipping the post-hoc test after a significant result. Reporting “ANOVA was significant” without following up leaves the actual research question — which fertilizer is best? — unanswered.
Practice Question
Three teaching methods are compared using test scores from 4 students each. SSB = 150, SSW = 60, k = 3 groups, N = 12 total students. Calculate F and state whether H₀ would be rejected at α = 0.05 (critical value ≈ 4.26 for df = 2, 9).
Answer: dfbetween = k−1 = 2. dfwithin = N−k = 12−3 = 9.
MSB = 150/2 = 75. MSW = 60/9 = 6.67.
F = 75 / 6.67 ≈ 11.24
Since 11.24 > 4.26, we reject H₀ — there is significant evidence that mean test scores differ across at least one of the three teaching methods. A post-hoc test would be needed next to identify which method(s) differ.
References
1. Moore, D.S., McCabe, G.P. and Craig, B.A. (2021) Introduction to the Practice of Statistics. W.H. Freeman.
2. Field, A. (2018) Discovering Statistics Using IBM SPSS Statistics. SAGE.
3. Fisher, R.A. (1925) Statistical Methods for Research Workers. Oliver and Boyd.
4. Triola, M.F. (2022) Elementary Statistics. Pearson.
5. Tukey, J.W. (1949) ‘Comparing Individual Means in the Analysis of Variance’, Biometrics, 5(2), pp. 99–114.