Step 5: Inference in Regression: t-Tests and Confidence Intervals
- Under the classical assumptions (SLR.1–SLR.6, with normal errors), \( t = (\hat\beta_1 – \beta_1)/\text{se}(\hat\beta_1) \) follows a \( t \) distribution with \( n-2 \) degrees of freedom.
- To test \( H_0: \beta_1 = c \), compute \( t = (\hat\beta_1 – c)/\text{se}(\hat\beta_1) \) and compare it with a critical value, or reject when \( p < \alpha \). The \( t \) printed by software tests only \( c = 0 \).
- A 95% confidence interval is \( \hat\beta_1 \pm t_{0.025,\,n-2}\,\text{se}(\hat\beta_1) \): exactly the values a two-sided 5% test would not reject.
- Statistical significance measures precision; economic significance measures size. Report both.
- For a 2,000-square-foot house the 95% interval for the average price is $389,400 to $404,100, but for one house it is $238,700 to $554,800.
From estimates to inference
In the companion house data, each extra square foot is associated with a price $140.76 higher. An estate agent says the going rate is $150. Is the gap a real difference, or the kind of difference you would expect from the luck of which 500 houses happened to sell? In the wage data, one more year of school goes with wages about 10.3% higher. Is that consistent with a return of 8%? Step 4 gave the centre and spread of the OLS slope. To turn them into probabilities you need its whole shape, and that is what statistical inference in regression adds.
Chapter 5 of Econometrics Made Simple covers this in full, with Python, R and Stata code for every example.Get the textbook · $15
The normality assumption
SLR.6 (normality) says the error is independent of \( x \) and normally distributed, \( u \sim N(0, \sigma^2) \). It implies SLR.4 and SLR.5 and adds a bell shape; SLR.1–SLR.6 together are the classical linear model (CLM) assumptions. Why might errors be normal? The error collects many small, separate influences, and the central limit theorem (Step 2) says such sums tend towards normality. The argument is weaker for skewed variables such as wages and house prices; the last section shows that in large samples it hardly matters.
Because \( \hat\beta_1 = \beta_1 + \sum w_i u_i \) is a weighted sum of the errors, and a weighted sum of normals is normal, SLR.6 gives \( \hat\beta_1 \sim N\!\left(\beta_1,\ \sigma^2/\text{SST}_x\right) \).
The t statistic
You never know \( \sigma \), so you replace it with \( \hat\sigma \) and the standard deviation with the standard error. For the house regression, \( \hat\sigma = 80.33 \) and \( \text{SST}_x = 199{,}526{,}258 \), so \( \text{se}(\hat\beta_1) = 80.33/14{,}125 = 0.005687 \).
The \( t \) statistic counts how many standard errors the estimate lies from the hypothesised value. Dividing by an estimated quantity adds uncertainty, which fattens the tails relative to the normal. The degrees of freedom are \( n-2 \) because the residuals must satisfy two equations, one for each estimated parameter. With more than about 120 degrees of freedom the \( t \) and the normal are almost identical.
| Two-sided level (one-sided) | df 10 | df 18 | df 30 | df 60 | df 120 | df 498 | \( \infty \) |
|---|---|---|---|---|---|---|---|
| 10% (5%) | 1.812 | 1.734 | 1.697 | 1.671 | 1.658 | 1.648 | 1.645 |
| 5% (2.5%) | 2.228 | 2.101 | 2.042 | 2.000 | 1.980 | 1.965 | 1.960 |
| 1% (0.5%) | 3.169 | 2.878 | 2.750 | 2.660 | 2.617 | 2.586 | 2.576 |
Testing whether x matters: H₀: β₁ = 0
The most common question asked of a regression is whether \( x \) has any linear association with \( y \): \( H_0: \beta_1 = 0 \) against \( H_1: \beta_1 \ne 0 \). A test does not try to prove \( H_0 \); it asks whether the data are inconsistent with it. Every test follows five steps.
| Step | What you do |
|---|---|
| 1 | State \( H_0 \), \( H_1 \) and the significance level \( \alpha \), before looking at the results. |
| 2 | Compute \( t = (\hat\beta_1 – c)/\text{se}(\hat\beta_1) \). |
| 3 | Find the critical value from \( t_{n-2} \). |
| 4 | Reject \( H_0 \) if \( |t| \) exceeds it (or if \( p < \alpha \)). |
| 5 | Conclude in words, in terms of the economic question. |
For 500 houses, \( \hat\beta_1 = 0.140765 \) with \( \text{se} = 0.005687 \). Is floor area associated with price at the 5% level?
The \( p \)-value is the probability, computed assuming \( H_0 \) is true, of a \( t \) statistic at least as extreme as the one observed: \( p = 2P(T > |t|) \) for a two-sided test. Here it is so small that software prints 0.000 or <2e-16. It is not the probability that \( H_0 \) is true.
When the result is not significant. Take only the first 20 workers in wages.csv and regress the log wage on education: \( \hat\beta_1 = 0.119 \), \( \text{se} = 0.067 \), \( t = 1.770 \), \( p = 0.094 \). With 18 df the 5% critical value is 2.101, so we do not reject \( H_0: \beta_1 = 0 \) at 5% (we would at 10%). Yet the estimated return, 11.9% a year, is larger than the full-sample 10.3%. The problem is precision: the standard error is more than eight times the full-sample value of 0.0081.
Testing other values and one-sided alternatives
Theory and policy often supply a value other than zero: is the price $150 a square foot, is the return to school 8%, is an elasticity equal to \(-1\)? The only change is that \( t \) measures the distance from \( c \).
An estate agent claims houses sell for $150 per square foot. With price in $000, that is \( \beta_1 = 0.15 \). Test it at 5%.
The estimate is below $150, but only by 1.6 standard errors, a gap sampling variation produces about one time in ten. The data do not contradict the claim.
In the full wage sample, \( \hat\beta_1 = 0.1026 \) with \( \text{se} = 0.0081 \) and \( n = 1{,}000 \). Test \( H_0: \beta_1 = 0.08 \) at 5%.
Two warnings belong in any write-up. This tests the association; omitted ability probably biases the OLS return upwards (Step 4). And the wage errors are heteroskedastic, which makes the usual standard error unreliable (Step 11).
model.t_test("educ = 0.08") does it; note that its coef column still shows the estimate itself, 0.1026.One-sided alternatives. When theory says which direction to expect, put all of \( \alpha \) in one tail.
| Alternative | Reject \( H_0: \beta_1 = c \) if | \( p \)-value |
|---|---|---|
| \( \beta_1 \ne c \) | \( |t| > t_{\alpha/2,\,n-2} \) | \( 2P(T > |t|) \) |
| \( \beta_1 > c \) | \( t > t_{\alpha,\,n-2} \) | \( P(T > t) \) |
| \( \beta_1 < c \) | \( t < -t_{\alpha,\,n-2} \) | \( P(T < t) \) |
For the 20 workers, human-capital theory predicts a positive return, so test against \( H_1: \beta_1 > 0 \). The one-sided 5% critical value with 18 df is 1.734, and \( t = 1.770 > 1.734 \): now we reject, with one-sided \( p = 0.047 \). The same data are “significant” or not depending on the alternative, which is why you must choose it before seeing the estimate. A buyer who suspects the agent is overstating prices would test \( H_1: \beta_1 < 0.15 \): \( t = -1.624 \) against \(-1.648\), just failing to reject (\( p = 0.053 \)).
Practise until it sticks: Econometrics Made Simple Practice Questions has over 1,150 questions, each with a worked answer.Get the practice book · $9
Statistical and economic significance
“Significant” in statistics means only that an estimate is precise enough to be told apart from zero (or from \( c \)). It says nothing about whether the effect is big enough to matter. Since \( \text{se}(\hat\beta_1) \approx \hat\sigma/(s_x\sqrt{n}) \), multiplying the sample by 100 cuts the standard error tenfold and raises \( t \) tenfold, with the same estimate.
Tiny but potentially significant. Regress the log wage on distance to the nearest college: the slope is \(-0.00129\) (each extra kilometre goes with a wage 0.13% lower), with \( t = -1.36 \) and \( p = 0.17 \). In the actual sample it is not significant, but with ten times as many workers and the same estimate \( t \) would be about \( -1.36 \times \sqrt{10} \approx -4.3 \). Nothing about the size would have changed. Large but insignificant. The 20-worker return of 11.9% is economically large but imprecise. The right conclusion is “a possibly large effect, too imprecisely estimated to be sure”.
So talk about size. A 500-square-foot extension is associated with a price about $70,000 higher (\( 0.1408 \times 500 = 70.4 \) in $000). Four extra years of school are associated with wages about 51% higher (\( 100(e^{4 \times 0.1026}-1) \)). The American Statistical Association’s 2016 statement on \( p \)-values makes the same point: a \( p \)-value does not measure the size of an effect or the importance of a result.
Confidence intervals for β₁
A test asks about one value. A confidence interval reports all the values the data do not reject. Rearranging \( P(-t_{0.025} < (\hat\beta_1 – \beta_1)/\text{se} < t_{0.025}) = 0.95 \) gives:
Compute 95% confidence intervals for the price per square foot, the return to schooling and the 20-worker return.
What “95% confidence” means: the parameter is fixed and the interval is random. If you repeated the sampling many times and built an interval each time, about 95% of those intervals would contain \( \beta_1 \). For your one interval, \( \beta_1 \) is either inside or not.
Predicting: the mean response and a single outcome
At floor area \( x_0 \), the fitted value \( \hat y_0 = \hat\beta_0 + \hat\beta_1 x_0 \) answers two different questions. What is the average price of all houses of that size? And what will this particular house sell for? The point estimate is the same; the uncertainty is not, because a single house also has its own error \( u_0 \), which no amount of data can predict.
With \( \hat\beta_0 = 115.2163 \), \( \hat\beta_1 = 0.140765 \), \( \hat\sigma = 80.333 \), \( n = 500 \), \( \bar x = 1{,}803.706 \) and \( \text{SST}_x = 199{,}526{,}258 \), find 95% intervals for the mean price of 2,000-square-foot houses and for one such house.
The average is pinned down to about ±$7,400; one house could sell for anything from about $239,000 to $555,000.
houses.csv, Econometrics Made Simple, Chapter 5.The confidence band is a narrow “bow tie”, tightest at \( \bar x \) and flaring out as \( x_0 \) moves away, because a small slope error matters more far from the centre. The prediction band is wide and nearly parallel, dominated by the constant \( \hat\sigma^2 \) term, which does not shrink as \( n \) grows. In Python, get_prediction().summary_frame() returns both (mean_ci_* and obs_ci_*); in R, predict() takes interval = "confidence" or "prediction".
Reading and reporting regression output
| Python (statsmodels) | R (lm) | Meaning, house regression |
|---|---|---|
coef | Estimate | \( \hat\beta_0 = 115.22 \), \( \hat\beta_1 = 0.1408 \) |
std err | Std. Error | \( \text{se}(\hat\beta_1) = 0.005687 \) (Python prints 0.006) |
t | t value | coef / std err: tests \( \beta_j = 0 \) only; 24.751 |
P>|t| | Pr(>|t|) | two-sided \( p \) for \( \beta_j = 0 \); 0.000 means below 0.0005 |
[0.025 0.975] | confint() | 95% interval: 0.130 to 0.152 |
| — | Residual standard error | \( \hat\sigma = 80.33 \) on 498 df |
R-squared | Multiple R-squared | 0.552 |
F-statistic | F-statistic | with one regressor \( F = t^2 = 24.751^2 = 612.6 \) (Step 7) |
In a project, report estimates with standard errors in parentheses beneath them:
Our guide to reading regression output goes through every number in a full table.
Without normality: large samples
House prices and wages are skewed, so their errors are probably not normal. Then \( t \) does not exactly follow \( t_{n-2} \). But by the central limit theorem, \( \hat\beta_1 – \beta_1 = \sum w_i u_i \) is approximately normal in large samples, and \( \hat\sigma \) settles at \( \sigma \). So under SLR.1–SLR.5 alone, \( t \) is approximately \( N(0,1) \), and the usual tests and intervals are approximately right. Econometrics Made Simple checks this by simulating 10,000 samples in which \( H_0 \) is true. With normal errors a 5% test rejects close to 5% of the time even at \( n = 5 \). With very skewed errors it rejects 6.2% of the time at \( n = 5 \) and close to 5% by \( n = 30 \) to 100; one-sided tests are more sensitive (5.8% even at \( n = 500 \)). Prediction intervals are the exception: they depend on the shape of the single error \( u_0 \), and no theorem rescues them.
The textbook and the practice book together: the Econometrics Made Simple bundle for $19 (was $24).Both books · $19
Practise: 6 questions
Try each one on paper first, then check the worked answer.
EasyQ1. A regression with \( n = 42 \) gives \( \hat\beta_1 = 0.84 \) and \( \text{se}(\hat\beta_1) = 0.30 \). Test \( H_0: \beta_1 = 0 \) against a two-sided alternative at 5%. Show worked answer
df \( = 40 \); 5% critical value 2.021.
\( 2.80 > 2.021 \): reject \( H_0 \) (\( p = 0.008 \)). \( x \) is significantly associated with \( y \).
EasyQ2. With 60 degrees of freedom, \( \hat\beta_1 = 2.5 \) and \( \text{se} = 0.8 \). Build a 95% confidence interval and say whether \( \beta_1 = 1 \) is rejected at 5%. Show worked answer
\( 2.5 \pm 2.000 \times 0.8 = [0.90,\ 4.10] \).
1 lies inside, so \( \beta_1 = 1 \) is not rejected at 5%.
MediumQ3. A log-log demand regression on 200 observations gives a price elasticity of \( -0.82 \) with standard error 0.07. Test whether demand is unit elastic (\( \beta_1 = -1 \)) at 5%. Show worked answer
df \( = 198 \); critical value 1.972.
\( 2.571 > 1.972 \): reject unit elasticity (\( p = 0.011 \)). Demand is inelastic: less responsive to price than unit elastic.
MediumQ4. With 25 degrees of freedom, \( t = 1.75 \) for \( H_0: \beta_1 = 0 \). Do you reject at 5% against (a) \( H_1: \beta_1 \ne 0 \); (b) \( H_1: \beta_1 > 0 \)? Show worked answer
(b) One-sided critical value 1.708; \( 1.75 > 1.708 \): reject (\( p = 0.046 \)).
The alternative must be chosen before seeing the data.
MediumQ5. Using the house regression (\( \hat\beta_0 = 115.2163 \), \( \hat\beta_1 = 0.140765 \), \( \hat\sigma = 80.333 \), \( n = 500 \), \( \bar x = 1{,}803.706 \), \( \text{SST}_x = 199{,}526{,}258 \), critical value 1.965), find 95% intervals for the mean price of 3,000-square-foot houses and for one such house. Show worked answer
\( (3{,}000 – 1{,}803.706)^2/\text{SST}_x = 1{,}196.294^2/199{,}526{,}258 = 0.007173 \).
Mean: \( \text{se} = 80.333\sqrt{0.002 + 0.007173} = 7.694 \); interval \( 537.51 \pm 15.12 = [522.4,\ 552.6] \).
One house: \( \text{se} = 80.333\sqrt{1.009173} = 80.700 \); interval \( 537.51 \pm 158.6 = [379.0,\ 696.1] \).
The mean interval is twice as wide as at 2,000 sq ft, because 3,000 is far from \( \bar x \).
ExamQ6. For the 500 houses, the regression of \( \log(\text{price}) \) on \( \log(\text{sqft}) \) gives a slope of 0.6929 with standard error 0.0280. (a) Interpret the slope. [1] (b) Test whether the slope is significantly different from zero at 1%. [2] (c) Test whether price is proportional to floor area (\( \beta_1 = 1 \)) at 5%. [3] (d) Build a 95% confidence interval and use it to explain your answer to (c). [2] Show worked answer
(b) \( t = 0.6929/0.0280 = 24.7 \) [1], far above the 1% critical value of 2.586 (498 df): reject. [1]
(c) \( H_0: \beta_1 = 1 \) vs \( H_1: \beta_1 \ne 1 \). [1] \( t = (0.6929 – 1)/0.0280 = -11.0 \) (\(-10.95\) with unrounded figures). [1] \( |t| > 1.965 \): reject; price rises less than proportionally with size. [1]
(d) \( 0.6929 \pm 1.965 \times 0.0280 = [0.638,\ 0.748] \). [1] 1 lies well outside the interval, so the 5% test rejects it. [1]
Total: 8 marks.
Check yourself
1. The \( t \) value printed next to a coefficient tests
2. A 95% confidence interval for \( \beta_1 \) is [0.42, 0.88]. At 5%, which value is not rejected?
3. Compared with the confidence interval for the mean response, the prediction interval for one new outcome is
Your score: 0 out of 3
The short version
- Under SLR.1–SLR.6, \( (\hat\beta_1 – \beta_1)/\text{se}(\hat\beta_1) \sim t_{n-2} \); two degrees of freedom go on estimating \( \beta_0 \) and \( \beta_1 \).
- Test \( H_0: \beta_1 = c \) with \( t = (\hat\beta_1 – c)/\text{se} \); reject when \( |t| \) exceeds the critical value or \( p < \alpha \). Choose one- or two-sided first.
- Houses: $140.76 per square foot (\( t = 24.75 \)); the $150 claim is not rejected (\( t = -1.624 \), \( p = 0.105 \)). Wages: an 8% return is rejected (\( t = 2.79 \)); with 20 workers even zero is not rejected at 5%.
- Statistical significance is about precision; economic significance is about size.
- 95% interval: \( \hat\beta_1 \pm t_{0.025,\,n-2}\,\text{se} \). The mean-response interval narrows with \( n \); the prediction interval stays wide.
- In large samples the tests remain approximately valid without normality.
- Chapter 5 of Econometrics Made Simple has the Python, R and Stata code for every test and interval here, and a full set of exercises with answers.
Studying more than one subject? The Complete Book Bundle includes every Economics, Statistics, Mathematics and Econometrics book.
Further reading on EconTutorials
Related in-depth articles from our blog: