Step 7: F-Tests and Joint Hypotheses
- In multiple regression the \( t \) test works as before, with \( n-k-1 \) degrees of freedom. A claim about one combination of coefficients, such as \( \beta_3 = \beta_2 \), needs the covariance of the two estimates, or a reparameterised model.
- Several restrictions at once (“experience has no effect” when experience enters twice) need the \( F \) test: \( F = \dfrac{(\text{SSR}_r-\text{SSR}_{ur})/q}{\text{SSR}_{ur}/(n-k-1)} \).
- Correlated variables can be individually insignificant but jointly highly significant. Test related groups together before dropping them.
- The \( R^2 \) form of \( F \) is valid only when both models have the same dependent variable. The SSR form always works.
- Exam point: with one restriction, \( F = t^2 \). The \( F \) test always rejects in the right tail, even when the alternative is two-sided.
- How to run \( t \) tests in multiple regression, including against non-zero values and one-sided alternatives.
- How to test a single linear combination of coefficients, with the covariance or by reparameterising.
- How to test several restrictions jointly with the \( F \) statistic, in its SSR and \( R^2 \) forms.
- Why variables can be jointly significant but individually insignificant, and what the overall \( F \) test tells you.
- How to test general restrictions such as a unit elasticity, how to report a regression table, and how the LM test works.
The \( t \) test in multiple regression
A regression with several regressors answers many questions at once, and most of them are not about one coefficient. Does experience matter at all, when it enters as a quadratic? Do regions differ? Is a year with your current employer worth more than a year elsewhere? This step builds the tools for those questions. It starts from the \( t \) test of Step 5.
Add the normality assumption MLR.6 (\( u \) is independent of the \( x \)s and normally distributed) to MLR.1–MLR.5 of Step 6. Then
where \( k \) counts the regressors, not the intercept. The degrees of freedom are the observations minus the number of estimated coefficients. Without normality, the central limit theorem makes the \( t \) and \( F \) distributions approximately right in large samples. With 1,000 workers or 500 houses, normality is not a worry.
Chapter 7 of Econometrics Made Simple covers this in full: t tests, linear combinations, F tests in SSR and R² form, general linear restrictions and the LM test, with Python, R and Stata output.Get the textbook · $15
The running example, from Chapter 7 of Econometrics Made Simple, is a wage equation estimated on the 1,000 workers in wages.csv:
where expersq \( = \text{exper}^2/100 \). Dividing by 100 only rescales \( \beta_3 \) to make it easier to read. With \( n = 1{,}000 \) and \( k = 5 \), the degrees of freedom are 994. The 5% two-sided critical value is 1.962, almost the familiar 1.96. Every coefficient has \( |t| > 4 \); tenure, for example, has \( t = 4.869 \).
Earlier studies put the return to a year of schooling at 8%. Test \( H_0\colon \beta_1 = 0.08 \) against \( H_1\colon \beta_1 > 0.08 \) at 1%, given \( \hat\beta_1 = 0.10480 \) and \( \operatorname{se}(\hat\beta_1) = 0.007573 \).
The return to schooling in these data is significantly above 8%. Remember that ability bias pushes this coefficient up (Step 6), so this is a statement about the regression coefficient, not proof that the causal return is that high.
Testing one combination of coefficients
Human-capital theory distinguishes general skills from firm-specific ones. If firm-specific skills are rewarded, a year of tenure should be worth more than a year of general experience. In the model \( \log(\text{wage}) = \beta_0 + \beta_1\text{educ} + \beta_2\text{exper} + \beta_3\text{tenure} + \beta_4\text{female} + u \), the hypothesis is \( H_0\colon \beta_3 – \beta_2 = 0 \).
The estimates are \( \hat\beta_3 = 0.01578 \) (se 0.00319) for tenure and \( \hat\beta_2 = 0.00565 \) (se 0.00196) for experience, with estimated covariance \( -4.08 \times 10^{-6} \). Test \( H_0\colon \beta_3 = \beta_2 \).
Reject at 5% but not at 1%. A year of tenure is associated with about one percentage point more pay than a year of general experience. Ignoring the covariance would give a standard error of 0.00374 and \( t = 2.71 \): too confident.
The reparameterisation trick
Standard output does not print the covariance. A neat way round it is to rewrite the model so that the combination becomes one coefficient. Let \( \theta = \beta_3 – \beta_2 \), so \( \beta_3 = \theta + \beta_2 \). Substitute and collect terms:
Create totexp = exper + tenure and regress log wage on educ, totexp, tenure and female. The coefficient on tenure is \( \theta \): 0.0101 with \( t = 2.151 \), exactly the direct result. It is the same model written differently, so \( R^2 \) and the residuals do not change.
The \( F \) test for several restrictions
Does experience matter at all? It enters through two terms, so the null hypothesis is \( H_0\colon \beta_2 = 0 \) and \( \beta_3 = 0 \). Two separate \( t \) tests do not answer this. Each tests one coefficient with the other left free, and two tests at 5% each do not make a test with a 5% false-rejection rate. We need one test of both restrictions together.
The logic is about fit. Dropping variables can never lower the sum of squared residuals, so \( \text{SSR}_r \ge \text{SSR}_{ur} \). The question is whether the rise is larger than chance would produce.
where \( k \) and \( n-k-1 \) refer to the unrestricted model. Reject if \( F \) exceeds the critical value. A large \( F \) is evidence against \( H_0 \) and a small one is not, so the rejection region is always in the right tail.
The unrestricted wage model has \( \text{SSR}_{ur} = 197.9457 \) with 994 df. Dropping exper and expersq gives \( \text{SSR}_r = 203.9813 \). Test \( H_0\colon \beta_2 = \beta_3 = 0 \) at 5%.
Experience and its square are jointly significant: experience matters for wages, holding education, tenure and gender fixed.
The \( R^2 \) form
Because \( \text{SSR} = \text{SST}(1-R^2) \), and both models share SST when they explain the same variable,
For experience, \( R^2_{ur} = 0.2495 \) and \( R^2_r = 0.2267 \), so \( F = (0.0228/2)/(0.7505/994) = 15.10 \), the same as 15.15 apart from rounding of the \( R^2 \)s. This form is handy when you only have the \( R^2 \)s of two published regressions.
A test that does not reject
In houses.csv, do bedrooms and bathrooms affect price once floor area, age, distance and a pool are held fixed? The \( F \) test gives \( F = 1.74 \) with \( p = 0.177 \): no evidence that they do. Without sqft in the model, the same two variables have \( F = 190 \). They only look important because more rooms usually means a bigger house. An \( F \) test answers a ceteris paribus question, so its answer depends on what else is in the model.
Computing F statistics by hand is a standard exam question. The Practice Questions book has over 1,150 questions, every one with a worked answer.Get 1,150+ questions · $9
Jointly significant, individually insignificant
It is common, and at first puzzling, to find a group of variables none of which is significant alone, yet which are highly significant together. The usual cause is strong correlation among them. When variables move almost in step, the data cannot tell which one is doing the work, so the individual standard errors are large. But the data can be very clear that the group matters.
Add a cubic term, expercu \( = \text{exper}^3/10{,}000 \), to the wage equation. The \( p \)-values on exper, expersq and expercu are 0.154, 0.902 and 0.491. Should experience be dropped?
A student who dropped every variable with \( p > 0.05 \) would conclude, wrongly, that experience does not affect wages.
Overall significance and general restrictions
Every regression output reports one \( F \) test automatically. It tests whether none of the regressors helps, \( H_0\colon \beta_1 = \cdots = \beta_k = 0 \). The restricted model \( y = \beta_0 + u \) has \( R^2 = 0 \), so
For the wage equation, \( F = (0.2495/5)/(0.7505/994) = 66.1 \). This is a low bar: with large samples almost every model passes. An \( R^2 \) of only 0.05 with \( n = 1{,}000 \) and \( k = 3 \) still gives \( F = 17.5 \).
Restrictions that are not zeros
Economic theory often predicts values other than zero. Is the price of a house proportional to its size? In
\( \beta_1 \) is the elasticity of price with respect to floor area. One restriction, \( \beta_1 = 1 \), is a \( t \) test: \( \hat\beta_1 = 0.6558 \), se 0.0392, so \( t = -8.79 \). The matching \( F \) is \( 77.23 = (-8.79)^2 \).
Test \( H_0\colon \beta_1 = 1,\ \beta_2 = 0,\ \beta_3 = 0 \) (price proportional to size, rooms irrelevant). The unrestricted SSR is 16.5135 with 493 df.
A 10% larger house sells for only about 6.6% more: big houses are cheaper per square foot. The \( R^2 \) form would be badly wrong here, giving 237.7, because the restricted model explains a different variable.
Reporting results and the LM test
Papers and dissertations present several models side by side in a regression table: coefficients with standard errors in brackets, significance stars with their key, \( n \), \( R^2 \) or adjusted \( R^2 \), which controls are included, and the joint tests that matter. Chapter 7 reports three wage models. Two rows tell the story:
| (1) | (2) | (3) | |
|---|---|---|---|
| educ | 0.1051*** (0.0077) | 0.1048*** (0.0076) | 0.1056*** (0.0074) |
| tenure | 0.0218*** (0.0024) | 0.0154*** (0.0032) | 0.0156*** (0.0031) |
| \( F \): exper, expersq = 0 | 15.15 [0.000] | 13.27 [0.000] | |
| \( F \): region dummies = 0 | 4.72 [0.003] | ||
| \( R^2 \) | 0.2267 | 0.2495 | 0.2838 |
Column (1) omits experience; (2) is the main model; (3) adds urban, union, married and region dummies. The return to education hardly moves. The tenure coefficient falls from 0.0218 to 0.0154 once experience enters, because long tenure goes with long experience (correlation 0.652): omitted variable bias from Step 6. For a guided tour of every number in a table like this, read How to Read Regression Output.
The Lagrange multiplier (\( nR^2 \)) test
The LM test needs only the restricted model. If the excluded variables have no effect, the restricted residuals should be unrelated to them.
For experience, \( R^2_{\tilde u} = 0.02959 \), so \( LM = 29.59 \), far above the 5% \( \chi^2_2 \) value of 5.991. LM is close to \( q \) times \( F \) (\( 2 \times 15.15 = 30.3 \)), and the \( p \)-values are almost identical. The same logic reappears in the heteroskedasticity tests of Step 11 and the autocorrelation tests of Step 12.
Practise: 6 questions
Try each one on paper first, then check the worked answer.
EasyQ1. An unrestricted model with \( n = 120 \) and \( k = 5 \) has \( \text{SSR}_{ur} = 450 \). Dropping three variables raises the SSR to 480. Compute \( F \) and test at 5% (the critical value of \( F_{3,114} \) is 2.68). Show worked answer
\( F = \dfrac{(480-450)/3}{450/114} = \dfrac{10}{3.947} = 2.53 \).
\( 2.53 < 2.68 \): do not reject at 5% (\( p \approx 0.06 \)). The three variables are not jointly significant at 5%, though they are at 10%.
EasyQ2. A regression with \( k = 4 \) and \( n = 105 \) has \( R^2 = 0.30 \). Compute the overall \( F \) statistic and state its degrees of freedom. Show worked answer
Degrees of freedom: 4 and 100.
The 1% critical value is about 3.51, so at least one regressor matters.
MediumQ3. Two regressions for log wage on the same 1,000 workers: the unrestricted model has \( k = 6 \) and \( R^2 = 0.452 \); dropping two variables gives \( R^2 = 0.430 \). Compute \( F \). Why is the \( R^2 \) form valid here? Show worked answer
\( F = \dfrac{(0.452-0.430)/2}{(1-0.452)/993} = \dfrac{0.011}{0.000552} = 19.93 \).
The 1% critical value of \( F_{2,993} \) is about 4.63, so reject.
The \( R^2 \) form is valid because both models have the same dependent variable, log wage, and the same observations, so they share the same SST.
MediumQ4. Two coefficients are \( \hat\beta_1 = 0.42 \) (se 0.10) and \( \hat\beta_2 = 0.30 \) (se 0.08), with estimated covariance 0.004. Test \( H_0\colon \beta_1 = \beta_2 \) using the normal approximation. What would you get if you ignored the covariance? Show worked answer
\( t = 0.12/0.0917 = 1.31 \): below 1.96, so do not reject.
Ignoring the covariance: \( \sqrt{0.0164} = 0.128 \) and \( t = 0.94 \).
Here the covariance is positive, so ignoring it overstates the standard error. The conclusion is the same, but the number is wrong.
MediumQ5. In \( y = \beta_0 + \beta_1x_1 + \beta_2x_2 + u \), you want to test \( H_0\colon \beta_1 = 2\beta_2 \). Write down a regression whose coefficient on one variable directly gives \( \theta = \beta_1 – 2\beta_2 \) and its standard error. Show worked answer
Substitute: \( y = \beta_0 + (\theta + 2\beta_2)x_1 + \beta_2x_2 + u = \beta_0 + \theta x_1 + \beta_2(2x_1 + x_2) + u \).
Create \( z = 2x_1 + x_2 \) and regress \( y \) on \( x_1 \) and \( z \).
The coefficient on \( x_1 \) is \( \theta \); its \( t \) statistic tests \( H_0\colon \theta = 0 \).
ExamQ6. A house-price model with \( n = 200 \) and \( k = 7 \) includes three neighbourhood dummies, none with \( |t| > 1.7 \). (a) Dropping the three dummies raises the SSR from 58.1 to 62.4. Test whether the dummies are jointly significant at 5% (critical value of \( F_{3,192} \): 2.65). [3] (b) Explain how the dummies can be jointly significant when none is significant alone. [2] (c) The auxiliary regression for the LM test has \( R^2 = 0.0689 \). Compute LM and test at 5% (\( \chi^2_3 \) critical value 7.81). [2] Show worked answer
\( F = \dfrac{(62.4 – 58.1)/3}{58.1/192} = \dfrac{1.433}{0.3026} = 4.74 \). [A1]
\( 4.74 > 2.65 \): reject; neighbourhood matters jointly. [A1]
(b) The dummies are correlated with each other and with the other regressors, so their individual standard errors are large and each \( t \) test asks whether one dummy adds anything given the rest. [B1] The \( F \) test asks whether the group as a whole adds explanatory power, which it can do even when the data cannot pin down each coefficient. [B1]
(c) \( LM = nR^2 = 200 \times 0.0689 = 13.8 \). [M1] \( 13.8 > 7.81 \): reject, agreeing with the \( F \) test. [A1]
Total: 7 marks.
Check yourself
1. You test 4 exclusion restrictions in a model with \( n = 200 \) and \( k = 9 \). The \( F \) statistic has degrees of freedom
2. When is the \( R^2 \) form of the \( F \) statistic not valid?
3. With a single restriction, how are the \( t \) and \( F \) statistics related?
Your score: 0 out of 3
The short version
- Under MLR.1–MLR.6, \( (\hat\beta_j – \beta_j)/\operatorname{se}(\hat\beta_j) \sim t_{n-k-1} \); in large samples this holds approximately without normality.
- A combination such as \( \beta_3 – \beta_2 \) needs the covariance term in its standard error, or a reparameterised model. A year of tenure is worth about one point more than a year of experience (\( t = 2.15 \)).
- \( F = [(\text{SSR}_r-\text{SSR}_{ur})/q] \,/\, [\text{SSR}_{ur}/(n-k-1)] \). Experience is jointly significant (\( F = 15.15 \)); bedrooms and bathrooms are not, once floor area is held fixed (\( F = 1.74 \)).
- Correlated variables can be individually insignificant but jointly significant: the cubic in experience has \( F = 10.26 \). Test related groups jointly.
- The overall \( F \) test is a low bar. General restrictions are tested by building the restricted model; a unit price–size elasticity is rejected (\( F = 59.8 \)).
- The LM statistic \( nR^2_{\tilde u} \sim \chi^2_q \) is a large-sample alternative to \( F \).
- Chapter 7 of Econometrics Made Simple covers all of this in full, with code, tables and a complete set of exercises with answers.
Master t tests, F tests and linear restrictions with the full chapter and hundreds of worked questions. Get the textbook and the practice book together for $19 (was $24).Both books · $19
Studying more than one subject? The Complete Book Bundle includes every Economics, Statistics, Mathematics and Econometrics book we publish.
Further reading on EconTutorials
Related in-depth articles from our blog, and the neighbouring steps: