Statistics guide, Unit 5, step 19 of 22Previous step Next step

Step 19: Exploring Relationships

By the EconTutorials team. 10-minute read with 6 practice questions.

SyllabusAP Stats 5.1–5.5

In 30 seconds

Plot two quantitative variables in a scatterplot: explanatory variable on the x-axis, response on the y-axis. Describe its direction, form, strength and outliers.
The correlation r (between −1 and +1) measures the strength and direction of a straight-line relationship only.
The least-squares line \( \hat{y} = a + bx \) has slope \( b = \Sigma(x-\bar{x})(y-\bar{y}) \div \Sigma(x-\bar{x})^2 \) and intercept \( a = \bar{y} – b\bar{x} \). Predict only inside the data range; \( r^2 \) is the share of variation explained.
The main exam point: correlation is not causation. Look for a lurking variable, reverse causation or coincidence.

Video lesson coming soonA short animated lesson for this step will appear here.

Two variables at once: the scatterplot

Does studying more raise your quiz score? Does a higher price cut demand for coffee? These questions are about relationships: how two things vary together. To answer them, you measure two quantitative variables on the same individuals.

Usually one variable is thought to explain the other. Study hours might explain quiz scores, so hours is the input and score is the output.

Key termExplanatory and response variables: the explanatory variable is the input you think influences the other, plotted on the x-axis; the response variable is the outcome, plotted on the y-axis.

A scatterplot puts one dot per individual at its \( (x, y) \) position. It is to two variables what a histogram is to one: the first graph you draw. Describe it with four features, in order:

Direction: positive (the cloud slopes up), negative (slopes down) or none.
Form: roughly linear (a straight-line trend) or curved.
Strength: how tightly the points cluster around the pattern. A thin band is strong; a loose cloud is weak.
Outliers: points far from the overall pattern.

When describing, say the variables are “associated”, not that one “causes” the other. A scatterplot shows a pattern, not a reason.

The correlation coefficient r

“Strong” and “weak” are vague. The correlation coefficient puts a single number on the strength and direction of a straight-line relationship.

Key termCorrelation coefficient (r): a number between −1 and +1 that measures the direction and strength of the linear relationship between two quantitative variables.

\[ r = \frac{\Sigma (x – \bar{x})(y – \bar{y})}{\sqrt{\Sigma (x – \bar{x})^2 \cdot \Sigma (y – \bar{y})^2}} \]

Five properties to memorise:

\( r \) always lies between −1 and +1.
Its sign gives the direction: positive for upward, negative for downward.
Values near ±1 mean the points hug a straight line; values near 0 mean little linear relationship.
\( r \) has no units, and changing units (cm to inches) does not change it.
\( r \) measures only linear association. A strong curve can still have \( r \) near 0.

Worked example

Five students studied \( x = 1, 2, 3, 4, 5 \) hours and scored \( y = 3, 5, 4, 6, 7 \) out of 10. Find \( r \).

Step 1Means: \( \bar{x} = 15 \div 5 = 3,\quad \bar{y} = 25 \div 5 = 5 \)

Step 2Deviations: \( x – \bar{x} = -2, -1, 0, 1, 2 \); \( y – \bar{y} = -2, 0, -1, 1, 2 \)

Step 3Products: \( 4, 0, 0, 1, 4 \), so \( \Sigma(x-\bar{x})(y-\bar{y}) = 9 \)

Step 4Squares: \( \Sigma(x-\bar{x})^2 = 4+1+0+1+4 = 10 \); \( \Sigma(y-\bar{y})^2 = 4+0+1+1+4 = 10 \)

Step 5\( r = \dfrac{9}{\sqrt{10 \times 10}} = \dfrac{9}{10} = 0.90 \)

There is a strong positive linear relationship: more study hours go with higher scores. Keep the sums 9 and 10; the regression line reuses them.

Cautions about correlation

A perfect U-shaped curve can have \( r \approx 0 \), and a single stray point can change \( r \) dramatically. Anscombe’s quartet is four datasets with the same means, standard deviations, correlation (\( r = 0.82 \)) and regression line, yet completely different shapes.

Common mistakeReporting \( r \) without looking at the scatterplot. The number cannot tell you whether the form is linear, whether an outlier is steering everything, or whether a curve is hiding. Calculate \( r \), but let the picture have the final word.

The least-squares regression line

When a scatterplot looks linear, you can summarise it with a line and use it to predict. For any line, the vertical gap between a point and the line is a residual: actual \( y \) minus predicted \( \hat{y} \).

Key termLeast-squares regression line: the one line that makes the sum of the squared residuals as small as possible.

\[ \hat{y} = a + bx \qquad b = \frac{\Sigma (x – \bar{x})(y – \bar{y})}{\Sigma (x – \bar{x})^2} = r\,\frac{s_y}{s_x} \qquad a = \bar{y} – b\bar{x} \]

The hat on \( \hat{y} \) means “predicted”. The line always passes through the point of averages \( (\bar{x}, \bar{y}) \).

Worked example

Fit the least-squares line to the study-hours data, then predict the score for 4.5 hours.

Step 1Slope: \( b = \dfrac{9}{10} = 0.9 \)

Step 2Intercept: \( a = 5 – 0.9 \times 3 = 5 – 2.7 = 2.3 \)

Step 3Line: \( \hat{y} = 2.3 + 0.9x \). Check at \( x = 3 \): \( 2.3 + 2.7 = 5 = \bar{y} \)

Step 4Prediction: \( \hat{y} = 2.3 + 0.9 \times 4.5 = 2.3 + 4.05 = 6.35 \)

The slope says each extra hour of study is associated with about 0.9 more quiz points on average. The intercept, 2.3, is the predicted score at 0 hours; treat it with caution because 0 is at the edge of the data. Since 4.5 hours lies inside the observed range of 1 to 5 hours, the prediction of 6.35 points is reasonable.

Prediction and extrapolation

Using a line inside the range of your data is interpolation. Pushing it beyond that range is extrapolation, and that is where predictions go wrong. A relationship that is straight over the data may bend, flatten or reverse outside it.

Common mistakeExtrapolating past the data. Extend a toddler’s growth line into adulthood and you would predict a nine-foot-tall adult. A line fitted to 1 to 5 hours of study says nothing reliable about 20 hours. Only predict within, or very near, the range of \( x \) you observed.

How good is the fit? Residuals and r²

A line can always be drawn; whether it fits is a separate question. Two tools help.

A residual plot graphs residuals against \( x \). If the line has captured the pattern, the residuals look like a random, shapeless band around zero. A leftover curve, funnel or trend means a straight line was the wrong choice. For example, the student who studied 4 hours and scored 6 has predicted score \( 2.3 + 0.9 \times 4 = 5.9 \), so residual \( = 6 – 5.9 = +0.1 \): the line slightly under-predicted.

Key termCoefficient of determination (\( r^2 \)): the proportion of the variation in \( y \) that is explained by the linear relationship with \( x \).

With \( r = 0.90 \), \( r^2 = 0.90^2 = 0.81 \). About 81% of the variation in quiz scores is explained by hours studied; the other 19% comes from factors the line does not capture, such as aptitude, sleep or luck.

Outliers and influential points

A point with an extreme \( x \)-value has high leverage: like the end of a see-saw, it can tip the whole line. If removing one point noticeably changes the slope or intercept, that point is influential. A point unusual only in \( y \) usually moves the line very little.

Correlation is not causation

Across some European regions, the number of stork pairs correlates with the number of human births. Do storks deliver babies? No. Bigger regions have more of everything: more land, more chimneys for storks to nest on, more people and so more births. Area is a hidden third variable driving both counts.

A strong correlation between \( x \) and \( y \) fits several different stories:

\( x \) causes \( y \);
\( y \) causes \( x \) (reverse causation);
a third lurking variable causes both (confounding);
it is coincidence.

For example, a city finds months with more ice-cream sales also have more drownings (\( r = 0.85 \)). Banning ice cream would achieve nothing: hot weather drives both ice-cream sales and swimming. The correlation is real but not causal.

The gold standard for proving cause is a randomised controlled experiment, where the explanatory variable is assigned at random so lurking variables are balanced out. From observational data you can build a case, but never a proof.

Exam tipWhen interpreting a slope, use the variables’ names and units and the words “is associated with”, for example: “each extra weekly workout is associated with 1.5 fewer beats per minute, on average.” For any causal claim from observational data, name a specific lurking variable, explain how it affects both variables, and say that a randomised experiment would be needed.

Practise: 6 questions

Try each one on paper first, then check the worked answer.

EasyQ1. Rank these correlations from strongest to weakest: \( r = 0.3 \), \( r = -0.8 \), \( r = 0.55 \). Show worked answer

Strength depends on the size of \( r \), ignoring the sign.
\( |-0.8| = 0.8 \), then \( 0.55 \), then \( 0.3 \).
Order: \( -0.8 \), \( 0.55 \), \( 0.3 \). The negative sign only tells you the direction is downward.

EasyQ2. An actual value is \( y = 12 \) and the line predicts \( \hat{y} = 10 \). Find the residual and say whether the line over- or under-predicted. Show worked answer

Residual \( = y – \hat{y} = 12 – 10 = +2 \).
The residual is positive, so the actual value is above the line: the line under-predicted.

MediumQ3. Given \( \Sigma(x-\bar{x})(y-\bar{y}) = 48 \), \( \Sigma(x-\bar{x})^2 = 40 \) and \( \Sigma(y-\bar{y})^2 = 90 \), find \( r \) and \( r^2 \). Interpret \( r^2 \). Show worked answer

\( r = \dfrac{48}{\sqrt{40 \times 90}} = \dfrac{48}{\sqrt{3600}} = \dfrac{48}{60} = 0.80 \)
\( r^2 = 0.80^2 = 0.64 \)
About 64% of the variation in \( y \) is explained by the linear relationship with \( x \); 36% is left unexplained.

MediumQ4. Given \( \Sigma(x-\bar{x})(y-\bar{y}) = 30 \), \( \Sigma(x-\bar{x})^2 = 12 \), \( \bar{x} = 5 \) and \( \bar{y} = 20 \), find the least-squares line and check that it passes through \( (\bar{x}, \bar{y}) \). Show worked answer

Slope \( b = 30 \div 12 = 2.5 \).
Intercept \( a = 20 – 2.5 \times 5 = 20 – 12.5 = 7.5 \).
Line: \( \hat{y} = 7.5 + 2.5x \).
Check at \( x = 5 \): \( 7.5 + 12.5 = 20 = \bar{y} \). It passes through \( (5, 20) \).

MediumQ5. Children with bigger shoe sizes have better reading scores. Does having bigger feet improve reading? Explain. Show worked answer

No. The lurking variable is age. Older children have bigger feet, and older children have also had more years to learn to read. Age drives both variables, so the correlation is real but not causal.

ExamQ6. A gym records weekly workouts (\( x \)) and resting heart rate in beats per minute (\( y \)) for its members. It finds \( r = -0.68 \) and \( \hat{y} = 78 – 1.5x \). (a) Interpret the sign of \( r \) and the slope. (b) Predict the resting heart rate for 4 workouts a week. (c) Should the gym use the line for 25 workouts a week? (d) Can the gym claim workouts cause a lower heart rate? [6 marks] Show worked answer

(a) Negative \( r \): more workouts are associated with a lower resting heart rate [1]. Slope: each extra weekly workout is associated with about 1.5 fewer beats per minute on average [1].
(b) \( \hat{y} = 78 – 1.5 \times 4 = 78 – 6 = 72 \) beats per minute [1].
(c) No: 25 workouts is far outside the observed range, so this is unreliable extrapolation [1].
(d) No: the data is observational [1]. A lurking variable, such as overall fitness or age, could affect both workout frequency and heart rate; only a randomised experiment could show cause [1].

Check yourself

1. A scatterplot shows a perfect U-shaped curve. Its correlation r is most likely…

r measures only straight-line association, so a symmetric curve can give r near 0.

2. Which point is most likely to be influential in regression?

Leverage lives in the x-direction: extreme x-values can tip the whole line.

3. If r = 0.8, what percentage of the variation in y is not explained by x?

\( r^2 = 0.64 \), so \( 1 – 0.64 = 0.36 \), or 36%, is unexplained.

Your score: 0 out of 3

The short version

Plot first: explanatory on x, response on y; describe direction, form, strength and outliers.
\( r \) runs from −1 to +1, has no units, is sensitive to outliers and measures only linear association.
The least-squares line \( \hat{y} = a + bx \) passes through \( (\bar{x}, \bar{y}) \). Interpret the slope as the change in \( y \) per unit of \( x \), and do not extrapolate.
Check fit with a patternless residual plot and \( r^2 \); watch for high-leverage, influential points.
Correlation is not causation. The full chapter in Statistics Made Simple has 54 practice questions with worked answers.