- A residual is observed minus predicted, \(e = y – \hat y\). A residual plot graphs the residuals against \(x\) (or against \(\hat y\)) to check whether a straight line is a suitable model.
- Random scatter around zero, with no pattern and roughly even spread, means a linear model fits. A curve means the relationship is not linear; a fan shape means the spread is not constant.
- In our braking case, the line \(\hat y = -17.15 + 0.912x\) has \(r = 0.988\), yet the residuals run +, then −, then + in a clear U-shape: a straight line is the wrong model.
- The standard deviation of the residuals, \(s = \sqrt{\sum e^2/(n-2)}\), gives the typical size of a prediction error; here \(s \approx 3.7\) m.
- Exam point: a high \(r\) does not prove a line is appropriate. Always justify “linear model is appropriate” by describing the residual plot: no pattern, scattered randomly about zero.
The case: is braking distance a straight-line function of speed?
A driving-school instructor is preparing a road-safety lesson. On a closed test track, the same car is braked hard from eight different speeds, and the braking distance (from the moment the brakes are applied to the moment the car stops) is measured:
| Speed (km/h), \(x\) | 20 | 30 | 40 | 50 | 60 | 70 | 80 | 90 |
|---|---|---|---|---|---|---|---|---|
| Braking distance (m), \(y\) | 6 | 11 | 17 | 25 | 34 | 45 | 56 | 70 |
A student fits a least-squares line and gets \(r = 0.988\). “That’s nearly perfect,” he says, “so braking distance goes up by a fixed amount for every extra km/h.” He then uses the line to predict the braking distance at 100 km/h and gets about 74 m. (The figures are illustrative.)
Is he right? The correlation is very high, but correlation only tells you how closely points follow some line. It does not tell you whether a straight line is the right shape. The tool that answers that question is the residual plot.
The concept: residuals and what their pattern reveals
Once you have fitted a regression line \(\hat y = a + bx\), every data point has a residual: the vertical distance from the point to the line. A positive residual means the point is above the line (the model under-predicted); a negative residual means it is below (the model over-predicted). For a least-squares line, the residuals always add up to zero.[1]
The residuals are what the line failed to explain. If the line has captured the true pattern, what is left over should look like random noise: no trend, no curve, no change in spread. If the residuals show a pattern, the line has missed something systematic. A residual plot simply turns this idea into a picture: put \(x\) (or the predicted values \(\hat y\)) on the horizontal axis and the residuals on the vertical axis, with a horizontal line at zero.
Here \(y_i\) is the observed value for the \(i\)th point, \(\hat y_i\) is the value the line predicts at the same \(x_i\), and \(e_i\) is the residual. \(a\) and \(b\) are the intercept and slope of the least-squares line. \(s\), the standard deviation of the residuals, measures the typical size of a prediction error, in the same units as \(y\). We divide by \(n – 2\), not \(n\), because two quantities (the slope and the intercept) were estimated from the data.[2]
Statistics Made Simple explains residuals and residual plots with worked examples, diagrams and tables.Get the textbook · $12
There are four patterns to recognise in a residual plot:
- Random scatter around zero, roughly even width: the linear model is appropriate.
- A curve (U-shape or upside-down U): the relationship is non-linear. A transformation or a curved model is needed.
- A fan (spread grows or shrinks with \(x\)): the variability of \(y\) is not constant, so predictions are more precise in some places than others.
- One point far from the rest: an outlier in \(y\), worth checking for a recording error or an unusual case.
Why this method fits the case
The student’s problem is not calculating the line; it is deciding whether a line is the right model. A residual plot is the standard check because:
- It magnifies departures from linearity. On the original scatterplot the braking points look almost straight. Removing the linear trend makes the leftover pattern easy to see.
- It works with any amount of data. With eight points or eight thousand, the logic is the same.
- It shows where the model fails. We learn not just that the line is wrong, but for which speeds it under- or over-predicts.
- It is what examiners expect. “Is a linear model appropriate?” is answered from the residual plot, not from \(r\).
It is not enough on its own when:
- The sample is very small (say four points). Almost any pattern, or none, can appear by chance.
- You need to know whether the slope is statistically significant. That needs a t-test for the slope.
- Data were collected over time. Then you should also plot residuals in time order, to check that one error does not lead to the next.
Setting it up
The least-squares line for the braking data uses \(n = 8\), \(\bar x = 55\), \(\bar y = 33\), \(S_{xx} = 4200\) and \(S_{xy} = 3830\). So \(b = 3830/4200 \approx 0.9119\) and \(a = 33 – 0.9119 \times 55 \approx -17.15\). The line is \(\hat y = -17.15 + 0.912x\). Now compute a predicted value and a residual for each speed:
| Speed \(x\) (km/h) | Observed \(y\) (m) | Predicted \(\hat y\) (m) | Residual \(e = y – \hat y\) (m) |
|---|---|---|---|
| 20 | 6 | 1.08 | 4.92 |
| 30 | 11 | 10.20 | 0.80 |
| 40 | 17 | 19.32 | −2.32 |
| 50 | 25 | 28.44 | −3.44 |
| 60 | 34 | 37.56 | −3.56 |
| 70 | 45 | 46.68 | −1.68 |
| 80 | 56 | 55.80 | 0.20 |
| 90 | 70 | 64.92 | 5.08 |
| Sum | 264 | 264 | 0 |
The residuals add to zero, as they must for a least-squares line. That is a useful arithmetic check.
Solving it
Use the residuals in the table to decide whether a linear model is appropriate for braking distance against speed.
A linear model is not appropriate: the residual plot shows a clear curved pattern, even though \(r = 0.988\).
Calculate the standard deviation of the residuals for the braking line and interpret it.
When the line is used to predict braking distance, the predictions are typically off by about 3.7 m. But because the errors are patterned, they are not random: at the extremes the line is consistently too low.
For comparison, a courier records delivery distance \(x\) (km) and journey time \(y\) (minutes) for six trips: (10, 22), (15, 30), (20, 37), (25, 47), (30, 53), (35, 62). The least-squares line is \(\hat y = 5.96 + 1.594x\). Find the residuals and judge the fit. (The figures are illustrative.)
The residuals scatter randomly about zero and are small, so a linear model is appropriate; predictions are typically within about 1 minute.
Make reading residual plots automatic with the Practice Questions book: 1,569 questions, all fully worked.Get 1,569 questions · $9
What the answer tells us
Figure 1 plots the eight braking residuals against speed. The U-shape is unmistakable: the line sits too high in the middle and too low at both ends.
Why the high correlation misled the student. The braking points rise steadily, so a line captures most of their variation and \(r^2 \approx 0.977\). But “most of the variation” is not “the right shape”. The leftover 2.3% is not random noise; it is the curvature. The residual plot isolates exactly that part.
Consequences for prediction. At 100 km/h the line predicts about 74 m. Because the residuals at the top end are already large and positive, the true distance is very likely to be longer. A model that follows the curve does far better. For example, regressing \(\sqrt{y}\) on \(x\) gives residuals that scatter randomly about zero (all within ±0.04), and back-transforming its prediction at 100 km/h gives about 85 m. Using the straight line here would understate the danger of high speed by about 11 m. That is the practical cost of skipping the residual check.[3]
Reasonableness check. The residuals sum to zero. The largest residuals (about 5 m) are at the ends of the data, which is typical for a straight line fitted to a curve. And the context agrees: basic physics says braking distance is proportional to the square of speed, so doubling speed roughly quadruples it. That is a squared relationship, not a linear one.
Residuals against \(x\) or against \(\hat y\)? With one explanatory variable and a positive slope, both plots have the same shape; only the horizontal scale changes. With a negative slope, the plot against \(\hat y\) is the mirror image. Computer output usually shows residuals against fitted values, and that version generalises to models with several explanatory variables.[4]
For a full treatment of fitting lines and checking them, the Statistics Made Simple textbook explains residuals and residual plots with clear diagrams and worked examples. To practise calculating residuals, reading plots and writing the conclusions examiners want, try the Statistics Made Simple Practice Questions book.
Master regression and model checking with both Statistics Made Simple books for $16.Both books · $16
Practice questions
Give answers to three significant figures where appropriate.
EasyQ1. The regression line of a student’s exam score on revision hours is \(\hat y = 12.5 + 0.8x\). A student who revised for 40 hours scored 47. Find and interpret the residual. Show worked answer
Residual: \(e = 47 – 44.5 = 2.5\).
The student scored 2.5 marks more than the line predicted for 40 hours of revision.
EasyQ2. Describe what each residual plot pattern suggests: (a) random scatter about zero; (b) a fan shape that widens to the right; (c) an upside-down U-shape. Show worked answer
(b) The spread of \(y\) increases as \(x\) increases, so predictions are less precise for larger \(x\).
(c) The relationship is curved (it levels off or bends down); a linear model is not appropriate.
MediumQ3. Data: \(x\) = 2, 4, 6, 8, 10 and \(y\) = 5, 9, 10, 14, 17. The least-squares line is \(\hat y = 2.3 + 1.45x\). Calculate all five residuals and comment on the residual plot. Show worked answer
Residuals: \(-0.2\), \(0.9\), \(-1.0\), \(0.1\), \(0.2\). Sum \(= 0\) (check).
Signs −, +, −, +, + show no systematic pattern and the residuals are small, so a linear model appears appropriate.
MediumQ4. Using the residuals from Q3, calculate the standard deviation of the residuals \(s\) and interpret it. Show worked answer
\(s = \sqrt{1.90/(5 – 2)} = \sqrt{0.6333} \approx 0.796\).
Predictions of \(y\) from the line are typically off by about 0.8 units.
ExamQ5. For six students, hours of extra tuition \(x\) and test score \(y\) were (1, 48), (2, 55), (3, 60), (4, 62), (5, 63), (6, 63). The least-squares line is \(\hat y = 48.4 + 2.886x\) and \(r = 0.906\). (a) Calculate the residuals. (b) Describe the residual plot and state whether a linear model is appropriate. (c) A teacher uses the line to predict the score after 7 hours. Find the prediction and comment on it. [7] Show worked answer
Residuals: \(-3.29\), \(0.83\), \(2.94\), \(2.06\), \(0.17\), \(-2.71\). [2]
(b) Negative, then positive, then negative: an upside-down U-shape. [1] The relationship is curved (scores level off), so a linear model is not appropriate, despite \(r = 0.906\). [1]
(c) \(\hat y = 48.4 + 2.886 \times 7 \approx 68.6\). [1] This is extrapolation and, since scores have levelled off at about 63, the line is likely to over-predict; the prediction is unreliable. [1]
Frequently asked questions
Why do the residuals always add up to zero?
Because the least-squares line passes through \((\bar x, \bar y)\), the positive and negative residuals exactly balance. If your residuals do not sum to zero (allowing for rounding), there is an arithmetic error somewhere.
Can I tell whether a model is good just from the size of the residuals?
Size and pattern answer different questions. Small residuals (a small \(s\)) mean predictions are precise. A random pattern means the model has the right shape. A good model needs both, and a model can have one without the other.
What should I do if the residual plot shows a curve?
Try a transformation, such as taking logarithms or square roots of one variable, and fit a line to the transformed data. Then check the new residual plot. If it shows random scatter, the transformed model is appropriate. Our regression post introduces the wider family of models.
How is the residual plot used in inference for regression?
Before carrying out a t-test or confidence interval for the slope, you must check the conditions: a linear relationship, roughly constant spread and roughly normal residuals. The residual plot checks the first two, and a histogram or dot plot of the residuals checks the third. See our hypothesis testing guide for the general framework.
References
- Illowsky, B. & Dean, S. (2023) Introductory Statistics 2e. Houston, TX: OpenStax. https://openstax.org/details/books/introductory-statistics-2e
- Starnes, D. S. & Tabor, J. The Practice of Statistics. New York: W. H. Freeman.
- NIST/SEMATECH. e-Handbook of Statistical Methods. National Institute of Standards and Technology. https://www.itl.nist.gov/div898/handbook/
- Moore, D. S., McCabe, G. P. & Craig, B. A. Introduction to the Practice of Statistics. New York: W. H. Freeman.
