When your data is lying to you — and how to get the truth anyway
Imagine you want to know: does more education make you earn more? Simple — just look at the data. People with more years of schooling tend to earn more. Run the regression. Done.
Except… it’s not that simple. Because smarter, more driven people tend to both stay in school longer AND earn more — regardless of their education level. So when you see “more education → higher earnings” in your data, you can’t tell if that’s the education itself doing the work, or if it’s the underlying ability of the person that caused both.
This is the endogeneity problem — and it’s probably the single most common reason economists mistrust each other’s results. Your regression coefficient is picking up things you didn’t intend to measure. The estimate is biased.
Instrumental Variables (IV) is the fix. The idea is clever: find a variable that nudges people into more (or less) education for reasons completely unrelated to their ability. Then measure only the earnings effect of that nudge. Pure, clean causation.
Three main reasons:
In all three cases, the core problem is the same: your regressor is correlated with the error term. OLS is not just imprecise — it’s wrong in a systematic direction.
Think of education as having two parts:
IV isolates the “clean” part by using an instrument — a variable that shifts how much education people get, but has nothing to do with their underlying ability. Then it asks: “Among people whose education level was shifted by this external factor, how did their earnings change?” That’s a clean causal estimate.
IV estimator (Wald) = (Effect of Z on Y) / (Effect of Z on X)
= Cov(Z, Y) / Cov(Z, X)
Intuitively: how much does Y change per unit increase in Z? Divide by how much X changes per unit of Z. You get the effect of X on Y, holding everything else constant through the instrument.
When you have controls and potentially multiple instruments, you use 2SLS — which does the IV procedure in two explicit regression stages:
Stage 1 — Clean up X: Regress the endogenous variable X on your instrument(s) Z and all other control variables. Save the predicted values X̂. These represent only the “clean” exogenous variation in X driven by Z.
X̂ᵢ = π₀ + π₁Zᵢ + (controls) + vᵢ
Stage 2 — Estimate the effect: Regress Y on X̂ (not X) and controls. The coefficient on X̂ is your 2SLS estimate — clean of endogeneity bias.
Yᵢ = β₀ + β₁X̂ᵢ + (controls) + uᵢ
ivregress 2sls in Stata, or the ivreg package in R.
This is where econometrics becomes more like detective work than maths. A good instrument needs to satisfy two conditions:
The instrument must actually change your endogenous variable. You test this by looking at the first-stage F-statistic. As a rule of thumb: if F < 10, your instrument is too weak and your IV estimates may be badly biased. Aim for F > 10, ideally much higher.
The instrument should only affect the outcome through its effect on the endogenous variable. This is the hard part — and where most IV papers get attacked. You have to argue this with logic and institutional knowledge, not just statistics.
Joshua Angrist (who won the Nobel Prize in 2021) wanted to know: does serving in the military help or hurt your long-run earnings? The obvious problem: who joins the military is not random. High-ability people often volunteer. So a naive comparison of veterans vs non-veterans mixes the effect of military service with the selection effect of who chose to serve.
Angrist’s solution was brilliant: the Vietnam draft lottery. In 1969, the US randomly assigned draft numbers to birth dates. If your birthday’s number was low, you were more likely to be called up. Random assignment — like a natural experiment.
The instrument: your draft lottery number.
Results:
The sign completely flipped! Military service itself — once you remove the selection bias — actually reduced civilian earnings significantly. Probably because veterans missed years of civilian work experience and education. This result would never have emerged without IV.
Source: Angrist, J.D. (1990). Lifetime earnings and the Vietnam era draft lottery. American Economic Review, 80(3), 313–336.
Angrist & Krueger (1991) — Quarter of birth: US compulsory schooling laws force students to stay in school until age 16. Students born early in the year start school older and hit the leaving age after less education. Birth quarter → years of education (relevant). Birth quarter has no direct effect on wages (exclusion). Their IV estimate of the return to education: 7.5% per year.
Acemoglu, Johnson & Robinson (2001) — Colonial settler mortality: To identify whether good institutions cause economic growth, they used historical disease mortality rates in European colonies as an instrument for current institutional quality. Where mortality was high, Europeans built extractive institutions. Where it was low, they settled and built better institutions. This gave them a clean IV for institutions — and found institutions cause dramatically higher income levels.
Here’s something subtle that often gets skipped in introductory courses. When you run IV, you don’t get the average treatment effect for the whole population. You get the effect specifically for the people who changed their behaviour because of the instrument — the “compliers.”
In Angrist’s draft lottery study, the 15% earnings penalty is the effect for men who only served because they drew a low lottery number. Volunteers (who would have served regardless) and avoiders (who found ways out regardless) aren’t included in the estimate. This is the Local Average Treatment Effect (LATE) — local to the complier group.
This matters for policy. An IV estimate might not generalise to other populations or contexts — you need to think carefully about who the “compliers” are before drawing broad policy conclusions.
A researcher wants to study the effect of smoking on health. Why is OLS likely to give a biased estimate? Propose a valid instrument, and explain why it satisfies both the relevance condition and the exclusion restriction.
OLS bias: Smokers differ from non-smokers in unobserved ways — they may have higher stress levels, lower incomes, or less healthy lifestyles generally. All of these independently affect health. So OLS conflates the effect of smoking with the effect of these correlated characteristics. The estimate of smoking’s health impact is likely overstated (more negative) because smokers already had worse health trajectories.
Instrument: cigarette taxes.
Relevance: Higher taxes → higher cigarette prices → people smoke less. Strong, testable relationship. ✅
Exclusion restriction: Tax rates set by state or national governments are unlikely to directly affect your health outcomes through any channel other than changing smoking behaviour. They don’t improve your diet, exercise habits, or healthcare access. ✅
Evans and Ringel (1995) used cigarette taxes in exactly this way to identify smoking’s causal effect on infant birth weight — finding that smoking reduced birth weight significantly after removing the selection bias.
Cov(Z, Y) = −12.6 and Cov(Z, X) = 3.5, where Z is distance from nearest university (km), X is years of education, Y is log wages. (a) Calculate the IV estimate. (b) Interpret the result. (c) What would you worry about regarding the exclusion restriction?
(a) IV estimate = Cov(Z,Y)/Cov(Z,X) = −12.6/3.5 = −3.6
(b) A one-year increase in education reduces log wages by 3.6 — but wait, that’s negative and implausible! More likely interpretation: the sign convention matters. Living further from a university (higher Z) is associated with less education (negative Cov(Z,X) — actually here it’s positive, meaning distance increases education, which is wrong). Let’s reinterpret: if Cov(Z,X) = 3.5 means more distance → more education (backwards), there’s likely a measurement or sign issue. In practice, you’d expect Cov(Z,X) < 0. Assuming the instrument is: closer to university → more education, and higher education → higher wages, the correctly signed IV estimate would be positive, representing the return to education.
(c) Exclusion restriction worry: distance from a university might correlate with urban vs rural residence, which itself affects wages (urban wage premium), job opportunities, industry mix, and internet access. If being closer to a university means you live in a richer, more productive area, then distance affects wages directly — not just through education. This would violate the exclusion restriction and invalidate the instrument.
The omitted variable bias formula, the sign table, and how to work out whether your coefficient is too big or too small before you touch the data. With worked examples from the returns-to-education literature.
Game Theory · Core ConceptsZero-Sum vs Non-Zero-Sum GamesOne of the most fundamental distinctions in game theory — and one that determines whether cooperation is even possible between rational players.When two people negotiate a salary, does one person's gain always...
Fiscal policy explained simply — multipliers, crowding out, Ricardian equivalence, and the austerity debate. A beginner-friendly guide to how government spending and taxation work, why Europe’s 2010 austerity programme hurt more than expected, and how to evaluate fiscal stimulus.
The omitted variable bias formula, the sign table, and how to work out whether your coefficient is too big or too small before you touch the data. With worked examples from the returns-to-education literature.
Game Theory · Core ConceptsZero-Sum vs Non-Zero-Sum GamesOne of the most fundamental distinctions in game theory — and one that determines whether cooperation is even possible between rational players.When two people negotiate a salary, does one person's gain always...
Fiscal policy explained simply — multipliers, crowding out, Ricardian equivalence, and the austerity debate. A beginner-friendly guide to how government spending and taxation work, why Europe’s 2010 austerity programme hurt more than expected, and how to evaluate fiscal stimulus.