- The least squares regression line \(\hat y = a + bx\) is the straight line that makes the sum of the squared vertical distances from the points as small as possible.
- The slope is \(b = S_{xy}/S_{xx}\) (or \(b = r\,s_y/s_x\)), and the intercept is \(a = \bar y – b\bar x\), so the line always passes through \((\bar x, \bar y)\).
- In our café case, \(\hat y = -7.86 + 2.71x\): each extra degree Celsius goes with about 2.7 more cold drinks sold, and a 23 °C day predicts about 55 drinks.
- Predictions inside the range of the data (interpolation) are reasonable; predictions far outside it (extrapolation), such as 40 °C or 0 °C, are unreliable and can be absurd.
- Exam point: interpret the slope as a predicted change in \(y\) per one-unit increase in \(x\), in context, and say whether the intercept has a sensible meaning.
The case: how many cold drinks should the café prepare?
A small seaside café sells chilled drinks. The owner wants to plan how many to prepare each morning, using the weather forecast. Over eight summer days she records the day’s maximum temperature and the number of cold drinks sold:
| Day | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
|---|---|---|---|---|---|---|---|---|
| Max temperature (°C), \(x\) | 18 | 20 | 22 | 24 | 26 | 28 | 30 | 32 |
| Cold drinks sold, \(y\) | 41 | 48 | 50 | 58 | 61 | 70 | 72 | 80 |
Sales clearly rise with temperature, and the correlation is \(r \approx 0.993\). But the owner does not want a measure of strength; she wants a formula. If tomorrow’s forecast is 23 °C, how many drinks should she expect to sell? What about a rare 40 °C heatwave? (The figures are illustrative.)
The tool for this is the least-squares regression line. This post shows how to calculate it, how to read its slope and intercept, how to use it for predictions, and when those predictions stop being trustworthy.
The concept: the line of best fit, defined precisely
Many straight lines could be drawn “through” a scatterplot. To choose one, we need a rule. For each point, the line gives a predicted value \(\hat y\) (“y-hat”). The vertical gap between the actual value and the predicted value, \(y – \hat y\), is called the residual. A good line has small residuals.
Some residuals are positive (points above the line) and some are negative (points below). Simply adding them would let them cancel. So we square them first. The least-squares line is the one line that makes the sum of squared residuals as small as possible. Using calculus, this minimum can be found exactly, and it gives simple formulas for the slope and intercept.[1]
Here \(x\) is the explanatory variable (the one we use to predict) and \(y\) is the response variable (the one we want to predict). \(\hat y\) is the predicted value of \(y\). The slope \(b\) is the predicted change in \(y\) when \(x\) increases by one unit. The intercept \(a\) is the predicted value of \(y\) when \(x = 0\). \(S_{xy} = \sum(x – \bar x)(y – \bar y)\) and \(S_{xx} = \sum(x – \bar x)^2\) are the same sums used for the correlation coefficient; \(r\) is the correlation, and \(s_x\), \(s_y\) are the sample standard deviations. Because \(a = \bar y – b\bar x\), the line always passes through the point of means \((\bar x, \bar y)\).[2]
Some textbooks write the line as \(\hat y = b_0 + b_1 x\), and Cambridge papers often use \(y = a + bx\). The letters change; the method does not. Always state which variable is \(x\) and which is \(y\), with units.
Why this method fits the case
The least-squares line suits the café problem because:
- There is a clear explanatory variable. Temperature is known in advance from the forecast; sales are what we want to predict. The roles of \(x\) and \(y\) are not interchangeable.
- The pattern is linear. Sales rise by a fairly steady amount for each 2 °C step, with no curve.
- The relationship is strong. With \(r = 0.993\), the points lie close to a line, so predictions should be precise within the observed range.
- There are no outliers or influential points that would drag the line away from the main pattern.
It is not the right tool when:
- The scatterplot is curved, such as sales levelling off at very high temperatures. A straight line would systematically over- and under-predict.
- You want to predict \(x\) from \(y\). The regression of \(y\) on \(x\) minimises vertical distances only; predicting temperature from sales needs a different line (the regression of \(x\) on \(y\)).
- The prediction is far outside the range of the data. The line was fitted only between 18 °C and 32 °C.
- You want to prove causation. Regression describes an association; only a designed experiment can show cause and effect.
Setting it up
Collect the summary statistics. From the table, \(n = 8\), \(\sum x = 200\), \(\sum y = 480\), \(\sum x^2 = 5168\), \(\sum xy = 12\,456\) and \(\sum y^2 = 30\,054\).
| Quantity | Formula | Value |
|---|---|---|
| Mean temperature | \(\bar x = \sum x / n\) | 25 °C |
| Mean drinks sold | \(\bar y = \sum y / n\) | 60 |
| \(S_{xx}\) | \(\sum x^2 – (\sum x)^2/n\) | 5168 − 5000 = 168 |
| \(S_{xy}\) | \(\sum xy – (\sum x)(\sum y)/n\) | 12 456 − 12 000 = 456 |
| \(S_{yy}\) | \(\sum y^2 – (\sum y)^2/n\) | 30 054 − 28 800 = 1254 |
| Correlation | \(r = S_{xy}/\sqrt{S_{xx}S_{yy}}\) | 0.993 |
Before any calculation, decide the roles: \(x\) = maximum temperature in °C (explanatory), \(y\) = number of cold drinks sold (response).
Solving it
Find the equation of the least-squares regression line of drinks sold on temperature.
The regression line is \(\hat y = -7.86 + 2.71x\), where \(x\) is the maximum temperature (°C) and \(\hat y\) is the predicted number of cold drinks sold.
Interpret the slope and intercept, then predict sales for a 23 °C day.
On a 23 °C day the café should expect to sell about 55 cold drinks.
A heatwave is forecast with a maximum of 40 °C. Use the line to predict sales, and comment on the reliability.
The line gives about 101 drinks, but the owner should treat this as a rough guess, not a reliable forecast.
What the answer tells us
Figure 1 shows the eight days with the fitted line. Within the data range (solid part) the points sit very close to the line. The dashed part extends the line out to 40 °C, where there is no data at all to support it.
Why “least squares”? The residuals for the eight days are 0, 1.57, −1.86, 0.71, −1.71, 1.86, −1.57 and 1.00. They add to zero (a property of every least-squares line with an intercept), and their squares add to about 16.29. Any other line does worse: shifting the line up by just 0.1 drinks raises the total to about 16.37. That is the precise sense in which this is the line of best fit.[3]
How good is the fit? Squaring the correlation gives \(r^2 \approx 0.987\). About 98.7% of the variation in daily drink sales is accounted for by the linear relationship with temperature. The remaining 1.3% is due to other things: day of the week, cloud cover, a coach party arriving.
Reasonableness check. The slope should be roughly the rise over the run of the data. From 18 °C to 32 °C sales rose from 41 to 80, a rise of 39 over 14 degrees, or about 2.8 per degree. Our \(b = 2.71\) is close. And the line passes through \((25, 60)\), the point of means, as it must.
Interpolation versus extrapolation. Inside the 18–32 °C window, predictions are supported by nearby data. Outside it, the line is a guess. At 40 °C it predicts 101 drinks, which may be too high. At 10 °C it predicts about 19, and at 0 °C it predicts a negative number. The further you go, the more absurd the answers can become.[4]
Regression is not symmetric. If you wanted to estimate the temperature from the number of drinks sold, you would need the regression of \(x\) on \(y\): slope \(S_{xy}/S_{yy} = 456/1254 \approx 0.364\). Rearranging that line gives \(y = -8.75 + 2.75x\), close to but not the same as our line. The two lines differ because each minimises a different set of distances, and they meet at \((\bar x, \bar y)\). They coincide only when \(r = \pm 1\).
For more on fitting and using regression lines, the Statistics Made Simple textbook covers the least-squares method with diagrams and step-by-step examples. To practise calculating lines, interpreting slopes and judging predictions, the Statistics Made Simple Practice Questions book has many fully worked regression questions in exam style.
Practice questions
Give coefficients to at least three significant figures.
EasyQ1. The regression line of a car’s price (£ thousand) on its age (years) is \(\hat y = 18.4 – 1.6x\). Interpret the slope. Show worked answer
EasyQ2. Using the café line \(\hat y = -7.86 + 2.714x\), predict drinks sold on a 29 °C day. Is this interpolation or extrapolation? Show worked answer
29 °C is inside 18–32 °C, so this is interpolation.
MediumQ3. For a group of students, the correlation between hours of sleep \(x\) and reaction score \(y\) is \(r = 0.6\), with \(\bar x = 15\), \(s_x = 4\), \(\bar y = 70\), \(s_y = 10\). Find the regression line of \(y\) on \(x\). Show worked answer
\(a = \bar y – b\bar x = 70 – 1.5 \times 15 = 47.5\).
\(\hat y = 47.5 + 1.5x\).
MediumQ4. Data: \(x\) = 1, 2, 3, 4, 5 and \(y\) = 2, 4, 5, 7, 7. Find the least-squares line and predict \(y\) when \(x = 6\). Comment on the prediction. Show worked answer
\(S_{xx} = 4 + 1 + 0 + 1 + 4 = 10\).
\(S_{xy} = (-2)(-3) + (-1)(-1) + 0 + (1)(2) + (2)(2) = 6 + 1 + 0 + 2 + 4 = 13\).
\(b = 13/10 = 1.3\); \(a = 5 – 1.3 \times 3 = 1.1\). Line: \(\hat y = 1.1 + 1.3x\).
At \(x = 6\): \(\hat y = 1.1 + 7.8 = 8.9\). This is extrapolation (just beyond the data), so treat it with caution; the last two points also suggest the pattern may be levelling off.
ExamQ5. A firm records monthly advertising spend \(x\) ($ thousand) and sales \(y\) (thousand units) for 8 months: \(\sum x = 44\), \(\sum y = 290\), \(\sum x^2 = 284\), \(\sum xy = 1728\). (a) Find the equation of the regression line of \(y\) on \(x\). (b) Interpret the slope in context. (c) Estimate sales when spend is $6.5 thousand. (d) The data cover spends from $2k to $9k. Comment on using the line to predict sales for a spend of $20k. [8] Show worked answer
\(S_{xx} = 284 – 44^2/8 = 284 – 242 = 42\); \(S_{xy} = 1728 – (44)(290)/8 = 1728 – 1595 = 133\). [2]
\(b = 133/42 \approx 3.167\); \(a = 36.25 – 3.167 \times 5.5 \approx 18.83\). So \(\hat y = 18.83 + 3.167x\). [2]
(b) Each extra $1,000 of advertising is associated with about 3,170 more units sold, on average. [1]
(c) \(\hat y = 18.83 + 3.167 \times 6.5 \approx 39.4\) thousand units. [1]
(d) $20k is far outside the data range, so this is extrapolation; the prediction (about 82 thousand units) is unreliable because the linear pattern may not continue (for example, diminishing returns to advertising). [1]
Frequently asked questions
Why do we square the residuals instead of using their absolute values?
Squaring makes the problem easy to solve exactly with algebra or calculus, and it always gives a single best line. It also penalises large errors more heavily than small ones. Lines that minimise absolute residuals exist, but they are harder to calculate and are beyond most introductory courses.
Does the regression line always pass through the mean point?
Yes. Since \(a = \bar y – b\bar x\), substituting \(x = \bar x\) gives \(\hat y = \bar y\). This is a useful check on your arithmetic and a quick way to draw the line by hand.
What is the connection between the slope and the correlation?
\(b = r\,s_y/s_x\). So the slope and \(r\) always have the same sign, and if \(r = 0\) the line is flat. A large slope does not mean a strong correlation: it depends on the units. Our post on the basics of correlation covers the meaning of \(r\).
How do I know whether a straight line is a good model?
Look at the residuals. If a residual plot shows random scatter around zero, a line is appropriate. If it shows a curve or a fan shape, a straight line is not the best model. This is the next step after fitting the line; our regression post gives the wider picture.
References
- Illowsky, B. & Dean, S. (2023) Introductory Statistics 2e. Houston, TX: OpenStax. https://openstax.org/details/books/introductory-statistics-2e
- Starnes, D. S. & Tabor, J. The Practice of Statistics. New York: W. H. Freeman.
- NIST/SEMATECH. e-Handbook of Statistical Methods. National Institute of Standards and Technology. https://www.itl.nist.gov/div898/handbook/
- College Board. AP Statistics Course and Exam Description. New York: College Board.

