- A chi-square test checks whether observed counts differ from the counts you would expect if a hypothesis were true.
- The statistic is \( \chi^2 = \sum (O-E)^2/E \), where \(O\) is an observed count and \(E\) the expected count.
- The goodness of fit test compares one variable with a claimed distribution (degrees of freedom \(k-1\)). The chi square test of independence asks whether two categorical variables are related (degrees of freedom \((r-1)(c-1)\)).
- In a contingency table, each expected count is \(E = (\text{row total} \times \text{column total}) / \text{grand total}\).
- Exam point: a significant result tells you an association exists, not how strong it is or what causes it. Report an effect size such as Cramér’s V and check that expected counts are at least 5.
What is a chi-square test and when do you use it?
A chi-square test (written \(\chi^2\), pronounced “kye-square”) is a hypothesis test for categorical data: counts in categories such as “passed” or “failed”, or “employed, unemployed, inactive”. It is not for averages of measured quantities such as income; for those you use a t-test or regression.
Karl Pearson introduced the test in 1900 in the Philosophical Magazine.[1] His idea was simple. If your hypothesis is true, you can work out how many observations should fall in each category. You then measure how far the real counts are from those expected counts. If the gap is bigger than chance alone would usually produce, you reject the hypothesis.
There are two main versions you need to know:
| Goodness of fit test | Test of independence | |
|---|---|---|
| Question | Does one variable follow a claimed distribution? | Are two variables related? |
| Data layout | One row of counts across \(k\) categories | A contingency table with \(r\) rows and \(c\) columns |
| \(H_0\) | The population shares equal the claimed shares | The two variables are independent |
| Expected count | \(E_i = n \times p_i\) | \(E_{ij} = \dfrac{\text{row}_i \times \text{col}_j}{n}\) |
| Degrees of freedom | \(k – 1\) (minus 1 more for each parameter estimated) | \((r-1)(c-1)\) |
Both tests assume independent observations (each item counted once, in one cell), raw counts rather than percentages, and expected counts that are not too small.
The chi square formula and degrees of freedom
The same chi square formula works for both versions of the test:
\(O – E\) is the gap in one cell. Squaring stops positive and negative gaps cancelling. Dividing by \(E\) scales the gap: missing by 10 matters when you expected 15, not when you expected 1,000. Then you add across all cells.
The result is never negative and equals 0 only when every count matches exactly. Larger values are stronger evidence against \(H_0\), so the test is always right-tailed: reject \(H_0\) when \(\chi^2\) exceeds the critical value.
Expected frequencies
In a goodness of fit test, the expected count in category \(i\) is the sample size times the claimed proportion: \(E_i = n p_i\). In a test of independence, the expected count in row \(i\) and column \(j\) is
This is the multiplication rule for independent events: cell probability = row probability × column probability, times the grand total to give a count. The NIST/SEMATECH e-Handbook of Statistical Methods derives the formula in exactly this way.[4]
Degrees of freedom
For goodness of fit with \(k\) categories, the counts must add up to \(n\). Once \(k-1\) counts are known, the last one is fixed, so \(df = k – 1\). If you estimate any parameter of the claimed distribution from the same data (for example the mean of a Poisson distribution), you lose one further degree of freedom for each parameter.[3]
For a table with \(r\) rows and \(c\) columns, both the row totals and the column totals are fixed. Only \((r-1)(c-1)\) cells are free, so \(df = (r-1)(c-1)\).[4] A 2×3 table therefore has \(1 \times 2 = 2\) degrees of freedom.
Critical values
You compare your statistic with an upper-tail critical value. The values below come from the NIST/SEMATECH e-Handbook’s table of the chi-square distribution.[2]
| df | \(\alpha = 0.10\) | \(\alpha = 0.05\) | \(\alpha = 0.01\) | \(\alpha = 0.001\) |
|---|---|---|---|---|
| 1 | 2.706 | 3.841 | 6.635 | 10.828 |
| 2 | 4.605 | 5.991 | 9.210 | 13.816 |
| 3 | 6.251 | 7.815 | 11.345 | 16.266 |
| 4 | 7.779 | 9.488 | 13.277 | 18.467 |
| 5 | 9.236 | 11.070 | 15.086 | 20.515 |
| 6 | 10.645 | 12.592 | 16.812 | 22.458 |
Source: NIST/SEMATECH e-Handbook, section 1.3.6.7.4.[2] The table there is labelled by the probability below the critical value, so the 0.95 column gives the 5% significance level.
The chi-square distribution: a diagram in words
Under \(H_0\), and with large enough expected counts, the statistic follows (approximately) a chi-square distribution. Its shape depends only on the degrees of freedom. You will often be asked to sketch it.
The mean equals the degrees of freedom and the peak is at \(df – 2\) (for df of 2 or more). As df rises the curve moves right and becomes more symmetrical, which is why critical values grow with df: more cells means a bigger total gap is normal even when \(H_0\) is true.
Worked example 1: goodness of fit test
A goodness of fit test asks whether the pattern in one categorical variable matches a claim. Economists use it to check claimed market shares, whether sales are spread evenly across weekdays, or whether survey respondents match the population’s age profile.
Suppose an industry report claims that four brands of instant coffee have market shares of 40%, 30%, 20% and 10%. You survey 250 randomly chosen shoppers and record which brand each bought: A = 118, B = 70, C = 42, D = 20. At the 5% significance level, is the claim consistent with your data? (Illustrative data.)
In words: brand A sold more than the report predicted, but a gap this size would turn up about 12% of the time through sampling luck alone. The data do not give strong enough evidence to reject the claimed market shares. Notice that brand A contributes more than half of the total \(\chi^2\), so if a larger survey found the same pattern, brand A is where the claim would break first.
Worked example 2: chi square test of independence (2×3 table)
The chi square test of independence asks whether two categorical variables are associated. In economics and education research you might ask whether employment status is related to region, whether loan default is related to income band, or, as below, whether exam results are related to the way students were taught.
Suppose a college records the results of 300 first-year statistics students taught in three ways. Test at the 5% level whether exam outcome is independent of teaching mode. (Illustrative data.)
| Observed | Online | Blended | In-person | Row total |
|---|---|---|---|---|
| Passed | 50 | 82 | 108 | 240 |
| Failed | 25 | 18 | 17 | 60 |
| Column total | 75 | 100 | 125 | 300 |
In words: there is strong evidence that pass rates differ by teaching mode. To see where the difference lies, look at the largest contributions. The “Failed, Online” cell contributes 6.667 of the 11.783 on its own: 25 online students failed when only 15 were expected. Pass rates were 66.7% online (50/75), 82% blended (82/100) and 86.4% in-person (108/125). The test does not show that online teaching causes lower results. Students chose their mode, and weaker or busier students may have chosen online study.
The 5-expected-count rule, Yates’ correction and Cramér’s V
The rule of five
The chi-square distribution is only an approximation to the true sampling distribution of the statistic. The approximation is poor when expected counts are small, because one or two extra observations in a tiny cell can make \((O-E)^2/E\) explode. The usual rule is that every expected count should be at least 5.[3][9] William Cochran’s 1954 review in Biometrics gave a slightly looser guide for larger tables: no expected count below 1, and no more than about 20% of expected counts below 5.[5]
The rule is about expected counts: an observed 0 is fine if its expected count is 8. If the rule fails, merge neighbouring categories that make sense together (for example “aged 65–74” and “75+”), collect more data, or use an exact test such as Fisher’s exact test for a 2×2 table.
Yates’ continuity correction
Counts are whole numbers, but the chi-square distribution is continuous. In a 2×2 table (1 degree of freedom) this mismatch makes the ordinary statistic slightly too large, so you reject \(H_0\) a little too often. Frank Yates proposed a fix in 1934: subtract 0.5 from each absolute difference before squaring.[6]
For example, in a 2×2 table with counts 30, 20, 15 and 35 (\(n = 100\)), the expected counts are 22.5, 27.5, 22.5 and 27.5. Every \(|O-E|\) is 7.5. The ordinary statistic is 9.091, and the Yates-corrected statistic, using \(7.5 – 0.5 = 7\), is 7.919. Both exceed 3.841, so the conclusion is unchanged here, but the corrected value is smaller and more cautious. Many statisticians now regard the correction as over-cautious, so check which version your syllabus or software uses. R applies it to 2×2 tables by default.[10]
Cramér’s V: how strong is the association?
A p-value tells you whether an association is likely to be real. It does not tell you whether it is large. With a big enough sample, even a trivial difference becomes “significant”, because \(\chi^2\) grows in proportion to \(n\). Harald Cramér’s measure, set out in his 1946 textbook Mathematical Methods of Statistics, rescales \(\chi^2\) to lie between 0 and 1:[7]
\(V = 0\) means no association and \(V = 1\) means perfect association. For the teaching-mode example, \(k = \min(2,3) = 2\), so
Jacob Cohen’s widely used benchmarks for tables where \(k – 1 = 1\) treat 0.1 as small, 0.3 as medium and 0.5 as large.[8] So the association is statistically clear but small to moderate in size. For a 2×2 table, Cramér’s V is the same as the phi coefficient.
A published real-data example: wafer defects by shift
The NIST/SEMATECH e-Handbook, produced by the US National Institute of Standards and Technology, works through a real industrial data set. A total of 309 defects in semiconductor wafers were classified by defect type (A to D) and by the production shift that made the wafer.[4] The question is the same one a manager would ask in any firm: does the type of fault depend on which shift is working?
| Shift | Type A | Type B | Type C | Type D | Total |
|---|---|---|---|---|---|
| 1 | 15 (22.51) | 21 (20.99) | 45 (38.94) | 13 (11.56) | 94 |
| 2 | 26 (22.99) | 31 (21.44) | 34 (39.77) | 5 (11.81) | 96 |
| 3 | 33 (28.50) | 17 (26.57) | 49 (49.29) | 20 (14.63) | 119 |
| Total | 74 | 69 | 128 | 38 | 309 |
Observed counts with expected counts in brackets. Source: NIST/SEMATECH e-Handbook of Statistical Methods, section 7.4.5.[4]
NIST reports \(\chi^2 = 19.18\) with \((3-1)(4-1) = 6\) degrees of freedom.[4] The 5% critical value for 6 df is 12.592,[2] so independence is rejected: defect type does depend on shift. Re-computing from the published counts gives \(p \approx 0.004\) and Cramér’s V \(= \sqrt{19.18/(309 \times 2)} \approx 0.18\), a small-to-moderate association. The largest gaps are shift 2’s excess of type B defects (31 against 21.44) and shortage of type D (5 against 11.81). That tells an engineer where to look, which is exactly how you should use the cell contributions in an exam answer.
How to run a chi-square test in Excel and R
Excel: CHISQ.TEST
Excel’s CHISQ.TEST(actual_range, expected_range) returns the p-value, not the \(\chi^2\) statistic.[9] You must build the expected table yourself.
- Type the observed counts, for example the 2×3 table in cells B2:D3, with row totals in E2:E3, column totals in B4:D4 and the grand total in E4.
- Build the expected table in B7:D8. In B7 type
=$E2*B$4/$E$4and fill it across and down. The dollar signs keep the totals locked in place. - In any cell type
=CHISQ.TEST(B2:D3,B7:D8). For the teaching-mode data this returns about 0.0028. - To get the statistic itself, use
=CHISQ.INV.RT(p, df)on that p-value, or add up the cell contributions with=SUMPRODUCT((B2:D3-B7:D8)^2/B7:D8), which gives 11.783.
Excel sets the degrees of freedom from the range’s shape: \((r-1)(c-1)\) for a table, \(c – 1\) or \(r – 1\) for a single row or column.[9] If you estimated a parameter from the data, that df is wrong, so compute the statistic with SUMPRODUCT and use =CHISQ.DIST.RT(statistic, df) with the correct df.
R: chisq.test
R’s built-in chisq.test() handles both versions.[10]
Goodness of fit, using the coffee example:
chisq.test(x = c(118, 70, 42, 20), p = c(0.4, 0.3, 0.2, 0.1))
This prints X-squared = 5.8533, df = 3, p-value = 0.119, matching the hand calculation.
Test of independence, using the teaching-mode table:
tab <- matrix(c(50, 25, 82, 18, 108, 17), nrow = 2)res <- chisq.test(tab)res$expected
This prints X-squared = 11.783, df = 2, p-value = 0.002762, and res$expected shows the expected table (60, 80, 100 / 15, 20, 25). The argument correct = TRUE is the default, but R only applies Yates’ correction to 2×2 tables, so it has no effect here.[10] For a 2×2 table, add correct = FALSE if your course uses the uncorrected statistic. If any expected count is below 5, R prints the warning “Chi-squared approximation may be incorrect”.
Common mistakes and how to evaluate a chi-square result
Other errors examiners see every year:
- Counting people twice. If a survey question allows several answers, the categories overlap and the observations are not independent. The test is invalid.
- Testing paired data. “Before and after” answers from the same people are not independent samples. Use McNemar’s test instead.
- Forgetting to divide by \(E\). \(\sum (O-E)^2\) on its own is not the chi-square statistic.
- Claiming causation. A significant result shows association. Like correlation, it says nothing about which variable drives the other, or whether a third factor drives both.
- Reporting only “significant”. Report \(\chi^2\), df, the p-value and an effect size, for example: \(\chi^2(2, N = 300) = 11.78,\ p = 0.003,\ V = 0.20\).
Evaluation points for essays
Strengths: no assumption about the population’s shape, works for nominal data that cannot be averaged, and easy to compute and explain. Weaknesses: large samples make tiny effects significant, small expected counts make it unreliable, merging categories loses information, and it ignores order. For ordered categories such as income bands, a test for trend is usually more powerful. Weigh these points against the purpose of the study rather than just listing them.
Practice questions
EasyQ1. State the degrees of freedom for (a) a goodness of fit test with 6 categories and no estimated parameters, and (b) a test of independence on a 4×5 contingency table. Show worked answer
(a) \(df = k – 1 = 6 – 1 = 5\).
(b) \(df = (r-1)(c-1) = (4-1)(5-1) = 3 \times 4 = 12\).
EasyQ2. In a contingency table with a grand total of 200, a cell sits in a row whose total is 80 and a column whose total is 50. Find the expected frequency under independence. Show worked answer
\(E = \dfrac{80 \times 50}{200} = \dfrac{4000}{200} = 20\).
MediumQ3. A die is rolled 60 times. The faces 1 to 6 come up 8, 12, 9, 11, 6 and 14 times. Test at the 5% level whether the die is fair. Show worked answer
\(H_0\): each face has probability \(1/6\). \(H_1\): at least one face has a different probability.
Expected count for each face: \(60 \times 1/6 = 10\).
\(\chi^2 = \dfrac{(8-10)^2 + (12-10)^2 + (9-10)^2 + (11-10)^2 + (6-10)^2 + (14-10)^2}{10} = \dfrac{4+4+1+1+16+16}{10} = \dfrac{42}{10} = 4.2\).
\(df = 6 – 1 = 5\). Critical value at 5% = 11.070. Since \(4.2 < 11.070\), do not reject \(H_0\) (\(p \approx 0.52\)). There is no evidence that the die is biased.
MediumQ4. A 2×2 table of 100 consumers shows: bought the product after seeing the advert 30, did not buy 20; bought without seeing the advert 15, did not buy 35. Calculate the uncorrected \(\chi^2\), the Yates-corrected \(\chi^2\) and Cramér’s V. Is there an association at the 5% level? Show worked answer
Row totals: 50 and 50. Column totals: bought 45, did not buy 55. Expected counts: \(50 \times 45/100 = 22.5\) and \(50 \times 55/100 = 27.5\) in each row.
Every \(|O – E| = 7.5\). Uncorrected: \(\chi^2 = 2 \times \dfrac{7.5^2}{22.5} + 2 \times \dfrac{7.5^2}{27.5} = 5.000 + 4.091 = 9.091\).
Yates: \(\chi^2 = 2 \times \dfrac{7^2}{22.5} + 2 \times \dfrac{7^2}{27.5} = 4.356 + 3.564 = 7.919\).
\(df = 1\), critical value 3.841. Both statistics exceed it, so reject independence: buying is associated with seeing the advert.
\(V = \sqrt{9.091/(100 \times 1)} = 0.302\), a medium association by Cohen’s benchmarks.
ExamQ5. A government survey of 10,000 households finds an association between owning a home and voting in the last election: \(\chi^2 = 4.2\) on a 2×2 table, \(p = 0.040\). A newspaper headline says “Home ownership makes people vote”. Evaluate this claim. [8 marks] Show worked answer
Mark scheme (up to 8 marks):
- Correct reading of the test (1): \(4.2 > 3.841\) with 1 df, so the association is significant at 5% but not at 1% (critical value 6.635).
- Effect size (2): \(V = \sqrt{4.2/10000} \approx 0.02\), far below Cohen’s “small” benchmark of 0.1. With \(n = 10{,}000\), even a trivial difference becomes significant, so the association is real but very weak.
- Causation (2): the test shows association, not causation. Age and income are likely confounders: older and richer households are both more likely to own homes and more likely to vote.
- Validity (1–2): with a large sample, the rule of five is almost certainly met; the result depends on independent, random sampling and accurate self-reported voting, which tends to be over-reported.
- Judgement (1): the headline overstates the finding. A fair summary is “home owners were slightly more likely to vote”, and a regression controlling for age and income would be needed before any causal claim.
Frequently asked questions
What is the difference between a goodness of fit test and a test of independence?
A goodness of fit test compares one categorical variable with a claimed distribution, using \(df = k – 1\). A test of independence uses a two-way table to ask whether two categorical variables are related, using \(df = (r-1)(c-1)\). The chi square formula is the same in both.
What does a large chi-square value mean?
A large \(\chi^2\) means the observed counts are far from the counts expected under \(H_0\). If it exceeds the critical value for your degrees of freedom, the p-value is below your significance level and you reject \(H_0\).
Can you use a chi-square test with small samples?
Only if every expected count is at least 5 (or, for larger tables, no more than about 20% below 5 and none below 1). Otherwise merge categories, collect more data, or use Fisher’s exact test for a 2×2 table.
Does CHISQ.TEST in Excel give the chi-square statistic?
No. CHISQ.TEST returns the p-value. Calculate the statistic with SUMPRODUCT((O-E)^2/E) or convert the p-value with CHISQ.INV.RT.
References
- Pearson, K. (1900). “On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling”. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, Series 5, 50(302), 157–175. https://doi.org/10.1080/14786440009463897
- NIST/SEMATECH (2012). “1.3.6.7.4. Critical Values of the Chi-Square Distribution”. e-Handbook of Statistical Methods. National Institute of Standards and Technology. https://www.itl.nist.gov/div898/handbook/eda/section3/eda3674.htm
- NIST/SEMATECH (2012). “1.3.5.15. Chi-Square Goodness-of-Fit Test”. e-Handbook of Statistical Methods. National Institute of Standards and Technology. https://www.itl.nist.gov/div898/handbook/eda/section3/eda35f.htm
- NIST/SEMATECH (2012). “7.4.5. How can we compare the results of classifying according to several categories?” (contingency tables). e-Handbook of Statistical Methods. National Institute of Standards and Technology. https://www.itl.nist.gov/div898/handbook/prc/section4/prc45.htm
- Cochran, W. G. (1954). “Some methods for strengthening the common χ² tests”. Biometrics, 10(4), 417–451. https://doi.org/10.2307/3001616
- Yates, F. (1934). “Contingency tables involving small numbers and the χ² test”. Supplement to the Journal of the Royal Statistical Society, 1(2), 217–235. https://doi.org/10.2307/2983604
- Cramér, H. (1946). Mathematical Methods of Statistics. Princeton University Press. https://press.princeton.edu/books/paperback/9780691005478/mathematical-methods-of-statistics
- Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum Associates; reissued by Routledge. https://doi.org/10.4324/9780203771587
- Microsoft (n.d.). “CHISQ.TEST function”. Microsoft Support. https://support.microsoft.com/en-us/office/chisq-test-function-2e8a7861-b14a-4985-aa93-fb88de3f260f
- R Core Team (n.d.). “chisq.test: Pearson’s Chi-squared Test for Count Data”. R documentation, package stats. https://stat.ethz.ch/R-manual/R-devel/library/stats/html/chisq.test.html

