Instant PDF download, name-and-email checkout. Complete Book Bundle: every Economics, Statistics, Mathematics, Econometrics and Research Methods book
Statistics

The Least-Squares Regression Line: Slope, Intercept, Predictions and Extrapolation

Find the least squares regression line step by step: slope, intercept and predictions, plus why extrapolation is risky, with worked examples and practice.

In 30 seconds
  • The least squares regression line \(\hat y = a + bx\) is the straight line that makes the sum of the squared vertical distances from the points as small as possible.
  • The slope is \(b = S_{xy}/S_{xx}\) (or \(b = r\,s_y/s_x\)), and the intercept is \(a = \bar y – b\bar x\), so the line always passes through \((\bar x, \bar y)\).
  • In our café case, \(\hat y = -7.86 + 2.71x\): each extra degree Celsius goes with about 2.7 more cold drinks sold, and a 23 °C day predicts about 55 drinks.
  • Predictions inside the range of the data (interpolation) are reasonable; predictions far outside it (extrapolation), such as 40 °C or 0 °C, are unreliable and can be absurd.
  • Exam point: interpret the slope as a predicted change in \(y\) per one-unit increase in \(x\), in context, and say whether the intercept has a sensible meaning.

The case: how many cold drinks should the café prepare?

A small seaside café sells chilled drinks. The owner wants to plan how many to prepare each morning, using the weather forecast. Over eight summer days she records the day’s maximum temperature and the number of cold drinks sold:

Day12345678
Max temperature (°C), \(x\)1820222426283032
Cold drinks sold, \(y\)4148505861707280

Sales clearly rise with temperature, and the correlation is \(r \approx 0.993\). But the owner does not want a measure of strength; she wants a formula. If tomorrow’s forecast is 23 °C, how many drinks should she expect to sell? What about a rare 40 °C heatwave? (The figures are illustrative.)

The tool for this is the least-squares regression line. This post shows how to calculate it, how to read its slope and intercept, how to use it for predictions, and when those predictions stop being trustworthy.

The concept: the line of best fit, defined precisely

Many straight lines could be drawn “through” a scatterplot. To choose one, we need a rule. For each point, the line gives a predicted value \(\hat y\) (“y-hat”). The vertical gap between the actual value and the predicted value, \(y – \hat y\), is called the residual. A good line has small residuals.

Some residuals are positive (points above the line) and some are negative (points below). Simply adding them would let them cancel. So we square them first. The least-squares line is the one line that makes the sum of squared residuals as small as possible. Using calculus, this minimum can be found exactly, and it gives simple formulas for the slope and intercept.[1]

Key termLeast squares regression line: the line \(\hat y = a + bx\) that minimises \(\sum (y – \hat y)^2\), the sum of the squared vertical distances between the observed values of \(y\) and the values predicted by the line. It is also called the line of best fit or the regression line of \(y\) on \(x\).
\[ \hat y = a + bx, \qquad b = \frac{S_{xy}}{S_{xx}} = r\,\frac{s_y}{s_x}, \qquad a = \bar y – b\,\bar x \]

Here \(x\) is the explanatory variable (the one we use to predict) and \(y\) is the response variable (the one we want to predict). \(\hat y\) is the predicted value of \(y\). The slope \(b\) is the predicted change in \(y\) when \(x\) increases by one unit. The intercept \(a\) is the predicted value of \(y\) when \(x = 0\). \(S_{xy} = \sum(x – \bar x)(y – \bar y)\) and \(S_{xx} = \sum(x – \bar x)^2\) are the same sums used for the correlation coefficient; \(r\) is the correlation, and \(s_x\), \(s_y\) are the sample standard deviations. Because \(a = \bar y – b\bar x\), the line always passes through the point of means \((\bar x, \bar y)\).[2]

Statistics Made Simple textbook coverStatistics Made Simple explains the least-squares regression line with worked examples, diagrams and tables.Get the textbook · $15

Some textbooks write the line as \(\hat y = b_0 + b_1 x\), and Cambridge papers often use \(y = a + bx\). The letters change; the method does not. Always state which variable is \(x\) and which is \(y\), with units.

Why this method fits the case

The least-squares line suits the café problem because:

  • There is a clear explanatory variable. Temperature is known in advance from the forecast; sales are what we want to predict. The roles of \(x\) and \(y\) are not interchangeable.
  • The pattern is linear. Sales rise by a fairly steady amount for each 2 °C step, with no curve.
  • The relationship is strong. With \(r = 0.993\), the points lie close to a line, so predictions should be precise within the observed range.
  • There are no outliers or influential points that would drag the line away from the main pattern.

It is not the right tool when:

  • The scatterplot is curved, such as sales levelling off at very high temperatures. A straight line would systematically over- and under-predict.
  • You want to predict \(x\) from \(y\). The regression of \(y\) on \(x\) minimises vertical distances only; predicting temperature from sales needs a different line (the regression of \(x\) on \(y\)).
  • The prediction is far outside the range of the data. The line was fitted only between 18 °C and 32 °C.
  • You want to prove causation. Regression describes an association; only a designed experiment can show cause and effect.

Setting it up

Collect the summary statistics. From the table, \(n = 8\), \(\sum x = 200\), \(\sum y = 480\), \(\sum x^2 = 5168\), \(\sum xy = 12\,456\) and \(\sum y^2 = 30\,054\).

QuantityFormulaValue
Mean temperature\(\bar x = \sum x / n\)25 °C
Mean drinks sold\(\bar y = \sum y / n\)60
\(S_{xx}\)\(\sum x^2 – (\sum x)^2/n\)5168 − 5000 = 168
\(S_{xy}\)\(\sum xy – (\sum x)(\sum y)/n\)12 456 − 12 000 = 456
\(S_{yy}\)\(\sum y^2 – (\sum y)^2/n\)30 054 − 28 800 = 1254
Correlation\(r = S_{xy}/\sqrt{S_{xx}S_{yy}}\)0.993

Before any calculation, decide the roles: \(x\) = maximum temperature in °C (explanatory), \(y\) = number of cold drinks sold (response).

Solving it

Worked example

Find the equation of the least-squares regression line of drinks sold on temperature.

Step 1 Slope: \(b = S_{xy}/S_{xx} = 456/168 \approx 2.714\).
Step 2 Intercept: \(a = \bar y – b\bar x = 60 – 2.714 \times 25 \approx 60 – 67.86 = -7.86\).
Step 3 Equation: \(\hat y = -7.86 + 2.714x\).
Step 4 Check with the other formula: \(s_x = \sqrt{168/7} \approx 4.899\), \(s_y = \sqrt{1254/7} \approx 13.384\), and \(r\,s_y/s_x = 0.9935 \times 13.384/4.899 \approx 2.714\). The slopes agree.

The regression line is \(\hat y = -7.86 + 2.71x\), where \(x\) is the maximum temperature (°C) and \(\hat y\) is the predicted number of cold drinks sold.

Worked example

Interpret the slope and intercept, then predict sales for a 23 °C day.

Step 1 Slope: for each increase of 1 °C in maximum temperature, the predicted number of cold drinks sold increases by about 2.7.
Step 2 Intercept: at 0 °C, the line predicts about −7.9 drinks. Negative sales are impossible, and 0 °C is far below any temperature in the data, so the intercept has no practical meaning here. It simply positions the line.
Step 3 Prediction: \(\hat y = -7.86 + 2.714 \times 23 \approx -7.86 + 62.43 = 54.57\).
Step 4 23 °C lies inside the observed range (18 to 32 °C), so this is interpolation and is reasonably reliable.

On a 23 °C day the café should expect to sell about 55 cold drinks.

Worked example

A heatwave is forecast with a maximum of 40 °C. Use the line to predict sales, and comment on the reliability.

Step 1 \(\hat y = -7.86 + 2.714 \times 40 \approx 100.7\), so about 101 drinks.
Step 2 40 °C is 8 °C above the hottest day in the data. This is extrapolation: we have no evidence that the straight-line pattern continues.
Step 3 Practical reasons it may fail: in extreme heat fewer people may go out, the café may run out of stock or seats, or demand may level off.

The line gives about 101 drinks, but the owner should treat this as a rough guess, not a reliable forecast.

Statistics Made Simple Practice Questions coverMake fitting regression lines automatic with the Practice Questions book: 1,569 questions, all fully worked.Get 1,569 questions · $9

What the answer tells us

Figure 1 shows the eight days with the fitted line. Within the data range (solid part) the points sit very close to the line. The dashed part extends the line out to 40 °C, where there is no data at all to support it.

Café sales and the least-squares line (eight summer days)
Fitted line in data rangeObserved daysExtrapolated line
304050607080901001101418222630343842Maximum temperature (°C), xCold drinks sold, y40 °C: 101? (extrapolation)23 °C: about 55range of the data (18–32 °C)
Figure 1. The line ŷ = −7.86 + 2.71x fits the eight days closely between 18 °C and 32 °C. The dashed extension to 40 °C is extrapolation: no data support it.

Why “least squares”? The residuals for the eight days are 0, 1.57, −1.86, 0.71, −1.71, 1.86, −1.57 and 1.00. They add to zero (a property of every least-squares line with an intercept), and their squares add to about 16.29. Any other line does worse: shifting the line up by just 0.1 drinks raises the total to about 16.37. That is the precise sense in which this is the line of best fit.[3]

How good is the fit? Squaring the correlation gives \(r^2 \approx 0.987\). About 98.7% of the variation in daily drink sales is accounted for by the linear relationship with temperature. The remaining 1.3% is due to other things: day of the week, cloud cover, a coach party arriving.

Reasonableness check. The slope should be roughly the rise over the run of the data. From 18 °C to 32 °C sales rose from 41 to 80, a rise of 39 over 14 degrees, or about 2.8 per degree. Our \(b = 2.71\) is close. And the line passes through \((25, 60)\), the point of means, as it must.

Interpolation versus extrapolation. Inside the 18–32 °C window, predictions are supported by nearby data. Outside it, the line is a guess. At 40 °C it predicts 101 drinks, which may be too high. At 10 °C it predicts about 19, and at 0 °C it predicts a negative number. The further you go, the more absurd the answers can become.[4]

Regression is not symmetric. If you wanted to estimate the temperature from the number of drinks sold, you would need the regression of \(x\) on \(y\): slope \(S_{xy}/S_{yy} = 456/1254 \approx 0.364\). Rearranging that line gives \(y = -8.75 + 2.75x\), close to but not the same as our line. The two lines differ because each minimises a different set of distances, and they meet at \((\bar x, \bar y)\). They coincide only when \(r = \pm 1\).

Common mistakeInterpreting the slope as a certainty: “every extra degree causes 2.7 more drinks to be sold.” The slope is a predicted or average change, and this is observational data, so it does not prove cause and effect. Use “is predicted to” or “on average”.
Common mistakeRounding the slope too early. If you round \(b\) to 2.7 and then compute \(a = 60 – 2.7 \times 25 = -7.5\), predictions change: at 23 °C you get 54.6 rather than 54.57, and at 40 °C you get 100.5 rather than 100.7. Keep at least four significant figures until the final answer.
Exam tipWrite the equation with words or define the symbols: “\(\hat y = -7.86 + 2.71x\), where \(\hat y\) is the predicted number of drinks and \(x\) is the maximum temperature in °C.” Use \(\hat y\), not \(y\), to show it is a prediction. When asked about reliability, mention two things: whether the value is inside the data range, and how strong the correlation is.

For more on fitting and using regression lines, the Statistics Made Simple textbook covers the least-squares method with diagrams and step-by-step examples. To practise calculating lines, interpreting slopes and judging predictions, the Statistics Made Simple Practice Questions book has many fully worked regression questions in exam style.

Statistics Made Simple bundle coverMaster correlation and regression with both Statistics Made Simple books for $19.Both books · $19

Practice questions

Give coefficients to at least three significant figures.

EasyQ1. The regression line of a car’s price (£ thousand) on its age (years) is \(\hat y = 18.4 – 1.6x\). Interpret the slope. Show worked answer
For each additional year of age, the predicted price falls by £1.6 thousand (£1,600) on average.
EasyQ2. Using the café line \(\hat y = -7.86 + 2.714x\), predict drinks sold on a 29 °C day. Is this interpolation or extrapolation? Show worked answer
\(\hat y = -7.86 + 2.714 \times 29 = -7.86 + 78.71 = 70.85\), so about 71 drinks.
29 °C is inside 18–32 °C, so this is interpolation.
MediumQ3. For a group of students, the correlation between hours of sleep \(x\) and reaction score \(y\) is \(r = 0.6\), with \(\bar x = 15\), \(s_x = 4\), \(\bar y = 70\), \(s_y = 10\). Find the regression line of \(y\) on \(x\). Show worked answer
\(b = r\,s_y/s_x = 0.6 \times 10/4 = 1.5\).
\(a = \bar y – b\bar x = 70 – 1.5 \times 15 = 47.5\).
\(\hat y = 47.5 + 1.5x\).
MediumQ4. Data: \(x\) = 1, 2, 3, 4, 5 and \(y\) = 2, 4, 5, 7, 7. Find the least-squares line and predict \(y\) when \(x = 6\). Comment on the prediction. Show worked answer
\(\bar x = 3\), \(\bar y = 5\).
\(S_{xx} = 4 + 1 + 0 + 1 + 4 = 10\).
\(S_{xy} = (-2)(-3) + (-1)(-1) + 0 + (1)(2) + (2)(2) = 6 + 1 + 0 + 2 + 4 = 13\).
\(b = 13/10 = 1.3\); \(a = 5 – 1.3 \times 3 = 1.1\). Line: \(\hat y = 1.1 + 1.3x\).
At \(x = 6\): \(\hat y = 1.1 + 7.8 = 8.9\). This is extrapolation (just beyond the data), so treat it with caution; the last two points also suggest the pattern may be levelling off.
ExamQ5. A firm records monthly advertising spend \(x\) ($ thousand) and sales \(y\) (thousand units) for 8 months: \(\sum x = 44\), \(\sum y = 290\), \(\sum x^2 = 284\), \(\sum xy = 1728\). (a) Find the equation of the regression line of \(y\) on \(x\). (b) Interpret the slope in context. (c) Estimate sales when spend is $6.5 thousand. (d) The data cover spends from $2k to $9k. Comment on using the line to predict sales for a spend of $20k. [8] Show worked answer
(a) \(\bar x = 5.5\), \(\bar y = 36.25\). [1]
\(S_{xx} = 284 – 44^2/8 = 284 – 242 = 42\); \(S_{xy} = 1728 – (44)(290)/8 = 1728 – 1595 = 133\). [2]
\(b = 133/42 \approx 3.167\); \(a = 36.25 – 3.167 \times 5.5 \approx 18.83\). So \(\hat y = 18.83 + 3.167x\). [2]
(b) Each extra $1,000 of advertising is associated with about 3,170 more units sold, on average. [1]
(c) \(\hat y = 18.83 + 3.167 \times 6.5 \approx 39.4\) thousand units. [1]
(d) $20k is far outside the data range, so this is extrapolation; the prediction (about 82 thousand units) is unreliable because the linear pattern may not continue (for example, diminishing returns to advertising). [1]

Frequently asked questions

Why do we square the residuals instead of using their absolute values?

Squaring makes the problem easy to solve exactly with algebra or calculus, and it always gives a single best line. It also penalises large errors more heavily than small ones. Lines that minimise absolute residuals exist, but they are harder to calculate and are beyond most introductory courses.

Does the regression line always pass through the mean point?

Yes. Since \(a = \bar y – b\bar x\), substituting \(x = \bar x\) gives \(\hat y = \bar y\). This is a useful check on your arithmetic and a quick way to draw the line by hand.

What is the connection between the slope and the correlation?

\(b = r\,s_y/s_x\). So the slope and \(r\) always have the same sign, and if \(r = 0\) the line is flat. A large slope does not mean a strong correlation: it depends on the units. Our post on the basics of correlation covers the meaning of \(r\).

How do I know whether a straight line is a good model?

Look at the residuals. If a residual plot shows random scatter around zero, a line is appropriate. If it shows a curve or a fan shape, a straight line is not the best model. This is the next step after fitting the line; our regression post gives the wider picture.

References

  1. Illowsky, B. & Dean, S. (2023) Introductory Statistics 2e. Houston, TX: OpenStax. https://openstax.org/details/books/introductory-statistics-2e
  2. Starnes, D. S. & Tabor, J. The Practice of Statistics. New York: W. H. Freeman.
  3. NIST/SEMATECH. e-Handbook of Statistical Methods. National Institute of Standards and Technology. https://www.itl.nist.gov/div898/handbook/
  4. College Board. AP Statistics Course and Exam Description. New York: College Board.
Statistics Made Simple Complete Bundle cover showing both the textbook and practice book
This topic has a full chapter in Statistics Made Simple, with 1,569 worked practice questions in the bundle.Statistics Made Simple Complete Bundle — Textbook + 1,569 Practice Questions
Get it for $19
E

The EconTutorials team writes clear, exam-focused lessons in economics and statistics for A-Level, IB, AP and university students, with more than 13 years of classroom teaching behind every guide and book in the Made Simple series.