Step 13: Regression and Causal Questions
MatchesQuantitative dissertations · going further: the Econometrics guide and how to read regression output
In 30 seconds
- Simple regression fits a straight line \( \hat y = b_0 + b_1 x \): the slope \( b_1 \) is the predicted change in \( y \) for one more unit of \( x \).
- Multiple regression adds control variables, so each slope is read holding the other variables fixed.
- Adding a confounder as a control can shrink or flip a coefficient, revealing how much of the raw association was due to it.
- Do not control for a mediator (something on the path from cause to outcome) if you want the total effect.
- \( R^2 \) is the share of variation in \( y \) explained; a low \( R^2 \) does not make a coefficient wrong.
Video lesson coming soonA short animated lesson for this step will appear here.
Why regression?
A t-test compares two groups. But hours of work are a continuous variable, and you want to account for other differences between students at the same time. Regression handles both. It is the workhorse of quantitative research in economics, business, health and the social sciences.
Simple regression
\[ \widehat{\text{mark}} = b_0 + b_1 \times \text{hours} \]
Worked example
Suppose a survey of 300 students gives \( \widehat{\text{mark}} = 70.2 – 0.42 \times \text{hours} \), with \( R^2 = 0.09 \) and \( p < 0.001 \) for the slope. Interpret it.
InterceptA student who does no paid work is predicted to score 70.2%.
SlopeEach extra weekly hour of work is associated with a mark 0.42 points lower.
PredictionAt 20 hours: \( 70.2 – 0.42 \times 20 = 61.8\% \).
\( R^2 \)Hours explain 9% of the variation in marks; the other 91% comes from everything else.
Caution“Associated with”, not “causes”: students who work more may differ in other ways.
Chapter 13 of Research Methods Made Simple covers this in full, running every regression in Python and R with the real output printed.Get the textbook · $15
Multiple regression and control variables
Students from lower-income families may need to work more and may have less support for study. Family income is then a confounder. Add it to the model:
\[ \widehat{\text{mark}} = b_0 + b_1 \times \text{hours} + b_2 \times \text{income band} \]
Worked example
Suppose the estimates become \( \widehat{\text{mark}} = 64.0 – 0.25 \times \text{hours} + 1.5 \times \text{income band} \), where income band runs from 1 (lowest) to 5 (highest).
Step 1The hours coefficient falls from −0.42 to −0.25. Holding income band fixed, one more hour of work goes with a mark 0.25 points lower.
Step 2Part of the original −0.42 was really the effect of family income, which is related to both working and marks.
Step 3Predict for a student working 10 hours in income band 3: \( 64.0 – 2.5 + 4.5 = 66.0\% \).
Step 4Even now the result is not proven causal: other confounders (motivation, course, health) may remain.
Confounders, mediators and colliders
| Role | Definition | Example | Control for it? |
|---|---|---|---|
| Confounder | Causes both X and Y | Family income → work and marks | Yes |
| Mediator | On the path X → M → Y | Work → less study time → lower marks | Not if you want the total effect |
| Collider | Caused by both X and Y | Being selected for an “employability award” that rewards both work and high marks | No: controlling creates false associations |
Drawing a simple diagram of arrows between your variables before you run anything helps you decide which controls belong in the model.
Stronger causal designs
Regression with controls can only adjust for confounders you measured. When a causal answer matters, researchers look for designs that deal with unmeasured confounders as well:
- Randomised experiments (Step 5).
- Panel data with fixed effects, comparing each person with themselves over time (Econometrics step 15).
- Difference-in-differences, comparing changes in a treated and an untreated group (DiD explained).
- Instrumental variables and natural experiments (Econometrics step 13).
In your dissertationPresent a regression table with each model in a column (Model 1 without controls, Model 2 with controls), showing coefficients, standard errors in brackets, n and \( R^2 \). Then explain in words what changed between the models and why.
Common mistakeAdding every available variable as a control. Controlling for a mediator hides part of the effect you want; controlling for a collider creates bias. Choose controls from your causal reasoning, not because they are in the dataset.
Practise until it sticks: Research Methods Made Simple Practice Questions has 1,004 questions, each with a worked answer.Get the practice book · $9
Practise: 6 questions
Try each one on paper first, then check the worked answer.
EasyQ1. In \( \widehat{\text{sleep}} = 8.4 – 0.3 \times \text{screen hours} \), interpret the slope. Show worked answer
Each extra hour of screen time is associated with 0.3 fewer hours (18 minutes) of sleep.
EasyQ2. Using \( \widehat{\text{mark}} = 70.2 – 0.42 \times \text{hours} \), predict the mark for 12 hours. Show worked answer
\( 70.2 – 0.42 \times 12 = 70.2 – 5.04 = 65.16 \), about 65.2%.
MediumQ3. Adding a control changes a coefficient from −0.42 to −0.25. What does this tell you? Show worked answer
The control was related to both the explanatory variable and the outcome, so part of the original association reflected it. The remaining −0.25 is the association holding the control fixed.
MediumQ4. Classify as confounder or mediator for the effect of work on marks: (a) study hours; (b) parents’ education. Show worked answer
(a) Mediator: work may reduce study hours, which then lower marks. (b) Confounder: parents’ education can affect both the need to work and marks.
MediumQ5. A model has \( R^2 = 0.08 \). A student says the model is useless. Respond. Show worked answer
A low \( R^2 \) means many other factors affect the outcome, which is normal for marks. The coefficient can still be precisely estimated and important. \( R^2 \) measures prediction, not whether the coefficient of interest is correct.
ExamQ6. Explain why a multiple regression of marks on hours worked may still not show the causal effect of work. [5] Show worked answer
It controls only for measured variables [1]; unmeasured confounders such as motivation or health may remain [1]. Reverse causality: students struggling academically may take more work or drop study [1]. Measurement error in self-reported hours biases the estimate [1]. Stronger designs (experiments, panel data, natural experiments) would help [1].
Check yourself
1. In multiple regression each slope is interpreted
That is what controlling means.
2. A variable on the causal path from X to Y is a
X → M → Y.
3. \( R^2 = 0.25 \) means
\( R^2 \) is the share of variance explained.
Your score: 0 out of 3
The textbook and the practice book together: the Research Methods Made Simple bundle for $19 (was $24).Both books · $19
The short version
- A regression slope is the predicted change in \( y \) for one more unit of \( x \).
- Controls let you hold confounders fixed; choose them from causal reasoning.
- Do not control for mediators or colliders.
- Regression with controls shows association; stronger designs get closer to causation.
Studying more than one subject? The Complete Book Bundle includes every Economics, Statistics, Mathematics, Econometrics and Research Methods book.
Want more questions like these, with worked answers?
The practice book has 1,004 more, from quick recall to exam style.