Hypothesis Testing in Economics
How do economists decide if something is actually true — or just a lucky coincidence?
Hypothesis testing only clicks once you have run the five steps yourself on new scenarios. The Statistics Made Simple Complete Bundle has dozens of worked tests with full mark schemes, plus 1,569 practice questions.
So… what even is hypothesis testing?
Okay, let’s start with a simple question. Say you flip a coin 10 times and get 7 heads. Is the coin rigged? Or did you just get lucky?
That’s basically what hypothesis testing does — it helps you figure out whether something you observed in data is a real pattern, or whether it could’ve just happened by chance.
In economics, we deal with this all the time. Did raising the minimum wage actually reduce jobs? Did a new training programme actually raise people’s earnings? Did a new tax actually change consumer behaviour? We can’t just look at the numbers and say “looks about right.” We need a proper way to test it.
Hypothesis testing is that proper way.
Think of it like a court case
Here’s a useful way to think about it. In a court, the accused is presumed innocent until proven guilty. You need strong enough evidence to convict.
Hypothesis testing works the same way:
- The null hypothesis (H₀) is “innocent” — no effect exists.
- Your data is the evidence.
- The significance level is the standard of proof you need.
- If the evidence is strong enough (p-value < 0.05), you “convict” — you reject the null and conclude something real is happening.
And just like in a court — failing to convict doesn’t mean the person is definitely innocent. It just means you didn’t have enough evidence. Similarly, failing to reject H₀ doesn’t mean the effect doesn’t exist. It just means your data couldn’t prove it.
The 5 steps every economist follows
- State your hypotheses. Write out H₀ and H₁ clearly. Example: H₀: the new drug has no effect on blood pressure. H₁: it does have an effect.
- Pick your significance level. Usually α = 0.05 (5%).
- Calculate your test statistic. This tells you how unusual your results are.
- Find the p-value. How likely are these results under H₀?
- Make your call. If p < 0.05 → reject H₀. If p ≥ 0.05 → don’t reject H₀.
The t-statistic — let’s do the maths (don’t panic!)
The most common test in economics is the t-test. The formula looks scarier than it is:
t = (what you observed − what you expected) / standard error
t = (x̄ − μ₀) / (s / √n)
Think of it this way: the t-statistic just asks “how many standard errors away from zero is my estimate?” If it’s far away (say, 2 or more), that’s suspicious. It suggests your result probably isn’t just random noise.
For a regression coefficient β̂:
t = β̂ / SE(β̂)
If |t| > 1.96, you reject H₀ at the 5% level. Simple as that.
Quick reference table — critical values
| Significance Level | Two-Tailed Cut-off | One-Tailed Cut-off |
|---|---|---|
| 10% (less strict) | ±1.645 | 1.282 |
| 5% (standard) | ±1.960 | 1.645 |
| 1% (strict) | ±2.576 | 2.326 |
Type I and Type II errors — the two ways to be wrong
Here’s something students always find tricky. There are two ways to mess up a hypothesis test:
| Reality: No effect | Reality: Effect exists | |
|---|---|---|
| You say: effect exists | ❌ Type I Error (false alarm) | ✅ Correct! |
| You say: no effect | ✅ Correct! | ❌ Type II Error (missed it) |
Confidence intervals — the range version of hypothesis testing
Instead of just saying “significant or not,” a confidence interval gives you a range where the true value probably sits.
95% CI = β̂ ± 1.96 × SE(β̂)
So if your estimate is 0.08 and your SE is 0.03, your 95% CI is (0.08 − 0.059, 0.08 + 0.059) = (0.021, 0.139).
If zero is NOT inside that interval, your result is statistically significant at 5%. If zero IS inside — you can’t rule out “no effect.”
Wait — what does the p-value actually mean?
This is probably the most misunderstood thing in all of statistics. Let’s clear it up.
A p-value of 0.03 means: “If there was truly no effect, there’s only a 3% chance we’d see results this extreme by pure luck.”
It does NOT mean:
- “There’s a 97% chance the effect is real.” (Nope.)
- “The effect is large or important.” (Nope.)
- “We found truth.” (Definitely nope.)
A big sample can give you a tiny p-value even for a completely meaningless effect. Always ask: “Okay, but is this effect actually big enough to matter in real life?”
📋 Case Study: Did Raising Minimum Wage Kill Jobs?
Card & Krueger (1994) — New Jersey Minimum Wage
For decades, economists said: “Raise the minimum wage → employers hire fewer people.” Seems logical, right? Higher costs → cut staff.
Then David Card and Alan Krueger came along and actually tested it. In 1992, New Jersey raised its minimum wage from $4.25 to $5.05. Neighbouring Pennsylvania kept it the same. They surveyed 410 fast-food restaurants on both sides of the border — before and after the change.
Their results:
- New Jersey fast-food employment actually went up slightly after the increase
- Pennsylvania employment went down slightly
- The difference: +4.61 workers per restaurant (SE = 1.61)
Let’s test it:
t = 4.61 / 1.61 = 2.86
p-value ≈ 0.004 → well below 0.05 → reject H₀
Conclusion: The minimum wage increase did NOT reduce employment — in fact it slightly increased it. This flipped the conventional wisdom on its head and launched a huge debate about labour market monopsony (where employers have market power over workers).
Card, D. & Krueger, A.B. (1994). American Economic Review, 84(4), 772–793.
✏️ Practice Questions
Question 1 — Try this yourself
A researcher finds that an extra year of schooling raises wages by 8.2% (SE = 0.031, n = 200). Test at the 5% level whether education affects wages.
👀 Show Answer
Step 1: t = 0.082 / 0.031 = 2.645
Step 2: Critical value at 5% (large sample) = 1.96
Step 3: 2.645 > 1.96 → Reject H₀
Step 4: 95% CI = 0.082 ± (1.96 × 0.031) = (0.021, 0.143) — doesn’t include zero ✓
Conclusion: Education has a statistically significant positive effect on wages at the 5% level. An extra year of schooling is associated with 8.2% higher pay.
Question 2 — Think it through
A job training programme raised earnings by £500/year (SE = £320, n = 40). Average earnings are £22,000. Is this result significant at 5%? And even if it isn’t, does that mean the programme was useless?
👀 Show Answer
t = 500 / 320 = 1.56. Critical value ≈ 2.02 (df = 39, two-tailed, 5%). Since 1.56 < 2.02 → fail to reject H₀. Not statistically significant.
But wait — the effect size is £500 / £22,000 = 2.3% earnings gain. That’s not trivial! The problem is the study only had 40 participants — too small to detect an effect this size. This is a Type II error risk — the real effect may exist but the sample is too small to prove it. The right response is to run a bigger study, not to conclude the programme doesn’t work.
Question 3 — Exam style
In your own words, explain the difference between a Type I and Type II error. Give one real-world example of each in an economic policy context.
👀 Show Answer
Type I Error (false positive): Concluding a policy works when it actually doesn’t. Example: a government study declares that a new apprenticeship scheme raises youth employment — but the effect was just seasonal variation. The government then wastes millions scaling up a programme that never worked.
Type II Error (false negative): Concluding a policy doesn’t work when it actually does. Example: an underpowered trial of a food voucher programme finds no significant effect on child nutrition. The programme gets cancelled. But the real effect was there — the study just wasn’t big enough to detect it, and thousands of children miss out on a benefit they would have received.
Neither error is always “worse” — it depends on the costs involved. Expensive policies make Type I errors costlier. Programmes targeting vulnerable groups make Type II errors costlier.
🎯 Summary — the key things to remember
- Hypothesis testing tells you whether a result is likely to be real or just a fluke.
- H₀ is your “nothing’s happening” baseline. H₁ is what you’re trying to show.
- The t-statistic measures how far your estimate is from zero, in standard error units.
- If p < 0.05, you reject H₀ and call the result statistically significant.
- Type I error = false alarm. Type II error = missed a real effect.
- Statistical significance ≠ economic importance. Always check the size of the effect too.
- The Card-Krueger minimum wage study is a brilliant example of data overturning theory.
📚 References & Further Reading
- Card, D. & Krueger, A.B. (1994). Minimum wages and employment. American Economic Review, 84(4), 772–793. — The classic study that started the minimum wage debate.
- Wooldridge, J.M. (2019). Introductory Econometrics (7th ed.). Cengage. — Best textbook for beginners going deeper.
- Stock, J.H. & Watson, M.W. (2020). Introduction to Econometrics (4th ed.). Pearson.
- Wasserstein et al. (2019). Moving beyond “p < 0.05.” The American Statistician, 73(S1). — A must-read on why p-values are often misused.
- Ziliak, S.T. & McCloskey, D.N. (2008). The Cult of Statistical Significance. University of Michigan Press. — Great accessible read on economic significance vs statistical significance.
