What 95% confident means

What you will be able to do

• say what "95% confident" means, and what it does not mean;
• compare intervals at the 90%, 95% and 99% levels;
• use the interval $\bar x \pm 1.96\dfrac{s}{\sqrt{n}}$ for the mean of a large sample.

A hundred surveys

In a made-up town, the true share of adults who own a bicycle is $p = 0.4$. Imagine that a hundred different teams each ask $100$ adults, chosen at random, and each team makes its own 95% interval. Each team has its own $\hat p$, so each interval sits in a slightly different place.

How many of the hundred intervals do you expect to catch $0.4$? Think, then look at the picture.

The picture shows exactly this, made by the program below. In this run, $94$ of the $100$ intervals catch the true share and $6$ miss. Another run might give $96$, or $93$. Over many runs, the share that catch settles near $95\%$.

What the level means

"We are $95\%$ confident" means this: the method catches the true value in about $95\%$ of samples. The confidence belongs to the method, like the record of a good archer.

Here are three readings that sound right but are not.

1. "There is a 95% chance that $p$ is in this interval." Once the interval is made, $p$ is either inside it or not. The $95\%$ describes what happens over many samples.
2. "95% of the people lie in the interval." The interval is about a parameter, such as the share or the mean. It is not about single values.
3. "95% of other samples will have $\hat p$ in this interval." Other samples are centred at $p$, not at our $\hat p$.

Other levels

Other levels use other multipliers, again from the normal curve.

A higher level gives a wider interval. To be more sure of catching the truth, you must cast a wider net. The only way to have both a high level and a narrow interval is a larger sample.

An interval for a mean

The same idea works for a mean. For a large random sample, the 95% confidence interval for a mean is

where $s$ is the sample standard deviation, with $n - 1$ in its formula. A made-up test of $64$ batteries gives a mean lifetime of $31.2$ hours with $s = 4.0$ hours. The standard error is $\dfrac{4.0\ \text{h}}{8} = 0.5$ hours, and the margin is $1.96 \times 0.5\ \text{h} = 0.98$ hours. So the interval runs from $30.22$ hours to $32.18$ hours.

For a small sample, statisticians use a slightly larger multiplier than $1.96$, taken from the Student t distribution. You will meet it in a later course. For samples of $30$ or more, $1.96$ is close.

Practise with new numbers

Try it in Python

The program runs the hundred surveys of the picture and counts the intervals that catch the true share.

Common mistakes

• Saying "a 95% chance that $p$ is in this interval". Say "we are $95\%$ confident", and mean the long-run record of the method.
• Reading the interval as the range of single values. It is a range for the parameter.
• Thinking a higher level is always better. A $99\%$ interval is wider and so says less.
• Using $1.96$ for a very small sample. Small samples need the Student t multiplier.

Remember these

• The confidence level is the long-run share of intervals that catch the true value.
• Multipliers: $1.645$ for $90\%$, $1.96$ for $95\%$, $2.576$ for $99\%$.
• For the mean of a large sample, the interval is $\bar x \pm 1.96\dfrac{s}{\sqrt n}$.

Check yourself

Try each one on your own first. If you are stuck, open a hint. Then check with Show the answer.

1. Fifty teams each make a 95% interval from their own random sample. About how many intervals do you expect to miss the true value?

2. A student says: "Our interval for the mean height is $160$ cm to $166$ cm, so $95\%$ of the students are between $160$ cm and $166$ cm tall." What is wrong?

3. For the same data, which interval is wider: the 90% one or the 99% one?

4. A random sample of $100$ apples has a mean mass of $152$ g and $s = 10$ g. Find the 95% interval for the mean mass.

5. Why can we not say that there is a $95\%$ chance that the true share lies in one particular interval?

References

• OpenStax, Introductory Statistics, Chapter 8, sections on a single population mean and on interpreting a confidence interval.