Prediction and the usual traps

What you will be able to do

• predict with a regression line and say when the prediction can be trusted;
• use $r^2$ and the residuals to judge a fit;
• recognise the traps: extrapolation, a single odd point, a hidden third variable and mixed groups.

Predicting

For the five students, the line $\hat{y} = 45 + 5x$ predicts a mark of $45 + 5 \times 3.5 = 62.5$ after $3.5$ hours of study. That hour value lies between $1$ and $5$, inside the data. This is interpolation, and it is usually safe.

Now ask for $12$ hours. The line gives $45 + 60 = 105$ marks, which is more than a full mark of $100$. This is extrapolation, a prediction outside the data. The line was fitted only for $1$ to $5$ hours. Outside that range we do not know that the pattern continues. It may bend or level off. Extrapolate with great care, and not far.

How good is the fit?

• The residuals. The residual of a point is $y - \hat{y}$. They add up to zero for the least squares line. A good fit has small residuals with no pattern when plotted.
• The coefficient of determination, $r^2$. The share of the variation in $y$ that is explained by the line of $y$ on $x$ is $r^2$. For the five students, $r^2 = 0.9766$, so about $97.7\%$ of the variation in the marks goes with the hours studied, and $2.3\%$ is left over. AP Statistics reports this share.
• The size of the misses. The smallest sum of squared residuals is $S_{yy}\left(1 - r^2\right)$. Here $256 \times 0.0234 = 6$.

Trap 1: one odd point

The left picture shows six points on a nearly perfect line, $r = 0.996$. Add one more point at $(7, 2)$, far below the line. Now $r = 0.285$, and the best-fit line is much flatter. One point can change the line and $r$ greatly, especially when it is far from the others in the $x$ direction. Look at the scatter plot, find such points and check them before you trust $r$.

Trap 2: correlation is not cause

Ice cream sales and the number of people who go swimming rise together in summer. Neither causes the other. A third variable, the temperature, drives both. Such a hidden variable is a lurking variable. A strong correlation shows an association only. To show cause, one needs an experiment that controls the other variables.

Trap 3: mixed groups

Each of two groups can show a rising pattern, while the two groups together show a falling pattern. This happens when one group has larger $x$ and smaller $y$ than the other. Look at the groups separately before you combine them.

Trap 4: other things

• A curved pattern. $r$ can be near $0$ for a perfect curve.
• A short range of $x$. If only a narrow range of $x$ is available, $r$ is usually small even for a true relationship.
• Swapping the variables. The line of $y$ on $x$ is not the line of $x$ on $y$, but $r$ is the same.
• Rounding too early in the sums.

A worked example

A car dealer records the age $x$ of six cars in months, and its value $y$ in hundreds of pounds: ages $10, 20, 30, 40, 50, 60$ and values $85, 78, 72, 63, 58, 49$.

1. The line. $S_{xy} = -1245$ and $S_{xx} = 1750$, so $b = -0.7114$. The means are $35$ and $67.5$, so $a = 67.5 + 0.7114 \times 35 = 92.4$. The line is $\hat{y} = 92.4 - 0.7114\,x$.
2. A prediction inside the data. At $35$ months, $\hat{y} = 92.4 - 24.9 = 67.5$, which is $6750$ pounds.
3. A prediction outside the data. At $100$ months, $\hat{y} = 21.26$. This is extrapolation. It may be wrong, because old cars do not always lose value at the same rate.
4. The fit. $r^2 = 0.9958$, so the line explains $99.6\%$ of the variation in the values.

Practise with new numbers

Try it in Python

The first program shows how a single odd point changes $r$. It also predicts marks from the hours at $3.5$, $5$ and $12$ hours, the last far outside the data.

Common mistakes

• Predicting far outside the data.
• Saying that $x$ causes $y$ because $r$ is high.
• Reading $r^2$ as $r$. An $r$ of $0.9$ explains $81\%$, not $90\%$.
• Ignoring an odd point in the scatter plot.
• Combining groups that differ, without looking at each one.

Remember these

• Interpolation, inside the data, is safer than extrapolation, outside it.
• $r^2$ is the share of the variation in $y$ explained by the line. The residuals add to zero.
• One odd point can change $r$ and the line a lot.
• Correlation does not show cause.

Check yourself

Try each one on your own first. If you are stuck, open a hint. Then check with Show the answer.

1. A line fitted to data with $x$ from $10$ to $40$ is used to predict $y$ at $x = 25$ and at $x = 90$. Which prediction is more reliable?

2. The correlation coefficient of two variables is $-0.8$. What share of the variation of $y$ is explained by the line?

3. A town finds that the number of drownings and ice cream sales are strongly correlated. What is the most likely explanation?

4. A line has the residuals $2, -3, 1$ and one more. What is the fourth residual?

5. Why can a strong $r$ from six points be misleading if one point is far from the others?

References

• OpenStax, Introductory Statistics, Chapter 12, sections on prediction, outliers and the coefficient of determination.
• A-Level, IB and AP Statistics teach interpolation and extrapolation, the lurking variable and the coefficient of determination $r^2$.