How do economists decide if something is actually true — or just a lucky coincidence?
Okay, let’s start with a simple question. Say you flip a coin 10 times and get 7 heads. Is the coin rigged? Or did you just get lucky?
That’s basically what hypothesis testing does — it helps you figure out whether something you observed in data is a real pattern, or whether it could’ve just happened by chance.
In economics, we deal with this all the time. Did raising the minimum wage actually reduce jobs? Did a new training programme actually raise people’s earnings? Did a new tax actually change consumer behaviour? We can’t just look at the numbers and say “looks about right.” We need a proper way to test it.
Hypothesis testing is that proper way.
Here’s a useful way to think about it. In a court, the accused is presumed innocent until proven guilty. You need strong enough evidence to convict.
Hypothesis testing works the same way:
And just like in a court — failing to convict doesn’t mean the person is definitely innocent. It just means you didn’t have enough evidence. Similarly, failing to reject H₀ doesn’t mean the effect doesn’t exist. It just means your data couldn’t prove it.
The most common test in economics is the t-test. The formula looks scarier than it is:
t = (what you observed − what you expected) / standard error
t = (x̄ − μ₀) / (s / √n)
Think of it this way: the t-statistic just asks “how many standard errors away from zero is my estimate?” If it’s far away (say, 2 or more), that’s suspicious. It suggests your result probably isn’t just random noise.
For a regression coefficient β̂:
t = β̂ / SE(β̂)
If |t| > 1.96, you reject H₀ at the 5% level. Simple as that.
| Significance Level | Two-Tailed Cut-off | One-Tailed Cut-off |
|---|---|---|
| 10% (less strict) | ±1.645 | 1.282 |
| 5% (standard) | ±1.960 | 1.645 |
| 1% (strict) | ±2.576 | 2.326 |
Here’s something students always find tricky. There are two ways to mess up a hypothesis test:
| Reality: No effect | Reality: Effect exists | |
|---|---|---|
| You say: effect exists | ❌ Type I Error (false alarm) | ✅ Correct! |
| You say: no effect | ✅ Correct! | ❌ Type II Error (missed it) |
Instead of just saying “significant or not,” a confidence interval gives you a range where the true value probably sits.
95% CI = β̂ ± 1.96 × SE(β̂)
So if your estimate is 0.08 and your SE is 0.03, your 95% CI is (0.08 − 0.059, 0.08 + 0.059) = (0.021, 0.139).
If zero is NOT inside that interval, your result is statistically significant at 5%. If zero IS inside — you can’t rule out “no effect.”
This is probably the most misunderstood thing in all of statistics. Let’s clear it up.
A p-value of 0.03 means: “If there was truly no effect, there’s only a 3% chance we’d see results this extreme by pure luck.”
It does NOT mean:
A big sample can give you a tiny p-value even for a completely meaningless effect. Always ask: “Okay, but is this effect actually big enough to matter in real life?”
For decades, economists said: “Raise the minimum wage → employers hire fewer people.” Seems logical, right? Higher costs → cut staff.
Then David Card and Alan Krueger came along and actually tested it. In 1992, New Jersey raised its minimum wage from $4.25 to $5.05. Neighbouring Pennsylvania kept it the same. They surveyed 410 fast-food restaurants on both sides of the border — before and after the change.
Their results:
Let’s test it:
t = 4.61 / 1.61 = 2.86
p-value ≈ 0.004 → well below 0.05 → reject H₀
Conclusion: The minimum wage increase did NOT reduce employment — in fact it slightly increased it. This flipped the conventional wisdom on its head and launched a huge debate about labour market monopsony (where employers have market power over workers).
Card, D. & Krueger, A.B. (1994). American Economic Review, 84(4), 772–793.
A researcher finds that an extra year of schooling raises wages by 8.2% (SE = 0.031, n = 200). Test at the 5% level whether education affects wages.
Step 1: t = 0.082 / 0.031 = 2.645
Step 2: Critical value at 5% (large sample) = 1.96
Step 3: 2.645 > 1.96 → Reject H₀
Step 4: 95% CI = 0.082 ± (1.96 × 0.031) = (0.021, 0.143) — doesn’t include zero ✓
Conclusion: Education has a statistically significant positive effect on wages at the 5% level. An extra year of schooling is associated with 8.2% higher pay.
A job training programme raised earnings by £500/year (SE = £320, n = 40). Average earnings are £22,000. Is this result significant at 5%? And even if it isn’t, does that mean the programme was useless?
t = 500 / 320 = 1.56. Critical value ≈ 2.02 (df = 39, two-tailed, 5%). Since 1.56 < 2.02 → fail to reject H₀. Not statistically significant.
But wait — the effect size is £500 / £22,000 = 2.3% earnings gain. That’s not trivial! The problem is the study only had 40 participants — too small to detect an effect this size. This is a Type II error risk — the real effect may exist but the sample is too small to prove it. The right response is to run a bigger study, not to conclude the programme doesn’t work.
In your own words, explain the difference between a Type I and Type II error. Give one real-world example of each in an economic policy context.
Type I Error (false positive): Concluding a policy works when it actually doesn’t. Example: a government study declares that a new apprenticeship scheme raises youth employment — but the effect was just seasonal variation. The government then wastes millions scaling up a programme that never worked.
Type II Error (false negative): Concluding a policy doesn’t work when it actually does. Example: an underpowered trial of a food voucher programme finds no significant effect on child nutrition. The programme gets cancelled. But the real effect was there — the study just wasn’t big enough to detect it, and thousands of children miss out on a benefit they would have received.
Neither error is always “worse” — it depends on the costs involved. Expensive policies make Type I errors costlier. Programmes targeting vulnerable groups make Type II errors costlier.
A false positive and a false negative — everyone mixes them up. The “boy who cried wolf” mnemonic that runs Type I then Type II in order, the trade-off nobody explains, why power = 1−β, and why which error to fear is a values judgement, not a formula.
The p-value is the probability of data this extreme assuming the null is true — not the probability the null is true, not the chance your result was luck, not the probability the alternative holds. The one correct definition, the three seductive wrong ones, and why significant doesn’t mean important.
Standard deviation describes your data; standard error describes your estimate. Here’s the difference, the SE = SD/√n formula that connects them, why only one shrinks with sample size, and the error-bar trap that catches everyone.
A false positive and a false negative — everyone mixes them up. The “boy who cried wolf” mnemonic that runs Type I then Type II in order, the trade-off nobody explains, why power = 1−β, and why which error to fear is a values judgement, not a formula.
The p-value is the probability of data this extreme assuming the null is true — not the probability the null is true, not the chance your result was luck, not the probability the alternative holds. The one correct definition, the three seductive wrong ones, and why significant doesn’t mean important.
Standard deviation describes your data; standard error describes your estimate. Here’s the difference, the SE = SD/√n formula that connects them, why only one shrinks with sample size, and the error-bar trap that catches everyone.
