Residuals and r squared

What you will be able to do

• draw and read a residual plot, and spot a bend, a fan or an outlier in it;
• work out $r^2$ and say what it means as a percentage;
• explain why a high $r$ is not enough to trust a straight line.

Stopping a car

On a made-up test track, a car brakes from different speeds, and the stopping distance is measured.

The least-squares line is $\hat{y} = -15.74 + 0.9383x$, and $r = 0.9916$. That looks excellent. Is a straight line right?

The residual plot

Work out the residual of every point and plot it against $x$. This is a residual plot. For the car, the residuals are about $5.0$, $1.6$, $-1.8$, $-3.2$, $-3.6$, $-2.9$, $-1.3$, $1.3$ and $4.9$ m.

They are positive at both ends and negative in the middle: a U shape. The points bend upwards, and the straight line cuts across the bend. A faster car needs much more distance than the line says. The true shape is a curve.

How to read a residual plot:

• A random band around $0$, with no pattern: a straight line is a good choice.
• A curve (a U or an upside-down U): the relationship is curved. Try another model.
• A fan that widens to one side: the predictions get less precise as $x$ grows.
• One point far from $0$: an outlier. Check it.

The residuals of a least-squares line always add up to $0$. So the plot is centred on the zero line. What matters is its shape.

How much does the line explain?

The values of $y$ vary. Their total variation about the mean is $S_{yy} = \sum (y - \bar{y})^2$. After the line, the variation left over is the SSE. The share that the line accounts for is

For a straight line fitted by least squares, this is exactly the square of $r$. For the car, $r^2 = 0.9916^2 = 0.983$: the line accounts for about $98.3\%$ of the variation in the stopping distances. For the canteen, $r^2 = 0.974$, about $97.4\%$.

$r^2$ is always between $0$ and $1$, and it has no sign. To go back from $r^2$ to $r$, take the square root and give it the sign of the slope.

Notice what happened with the car. $r^2 = 0.983$ is very high, but the residual plot shows that the line has the wrong shape. A high $r$ or $r^2$ does not prove that a straight line is right. The residual plot can show it is wrong.

Your turn: new numbers

Try it in Python

The program fits a line to the stopping distances, prints $r$ and $r^2$, and draws the residual plot.

The usual mistakes

• Trusting a high $r$ without the residual plot. A curve can give $r = 0.99$.
• Reading $r = 0.7$ as "the line explains $70\%$". It is $r^2 = 0.49$, so $49\%$.
• Losing the sign. $r^2 = 0.64$ gives $r = 0.8$ or $r = -0.8$. The slope decides.
• Expecting the residuals to be small. They should be patternless. Their sum is always $0$.

Remember these

• A residual plot shows the residuals against $x$. A random band means a straight line fits.
• $r^2 = 1 - \dfrac{\text{SSE}}{S_{yy}}$, the share of the variation in $y$ that the line accounts for.
• $r = \pm\sqrt{r^2}$, with the sign of the slope.

Check yourself

Try each one on your own first. If you are stuck, open a hint. Then check with Show the answer.

1. A line has $r = 0.6$. What percentage of the variation in $y$ does it account for?

2. A residual plot shows an upside-down U: negative at both ends and positive in the middle. What does it tell you?

3. A line slopes downwards and $r^2 = 0.81$. Find $r$.

4. Five residuals of a least-squares line are $2$, $-1$, $3$, $-2$ and one more. What is the last one?

5. For some data $S_{yy} = 60$, and the least-squares line has SSE $= 12$. Find $r^2$.

References

• OpenStax, Introductory Statistics, Chapter 12 (Linear Regression and Correlation), the sections on the coefficient of determination and on outliers.