Sample to sample: the scatter of a statistic

What you will be able to do

• tell a parameter of a population from a statistic of a sample;
• explain why a statistic changes from sample to sample;
• tell the scatter of chance from the steady error of a biased method.

Five friends, five answers

A made-up school has $1200$ students, and $420$ of them cycle to school. So the true share of cyclists is $\dfrac{420}{1200} = 0.35$. Five friends do not know this. Each friend asks $40$ students, chosen at random, and works out the share of cyclists in the sample.

Will the five friends get $0.35$? Think about it before you read on. The program below plays the five friends.

Try it in Python

The program takes five random samples of $40$ students from the school and prints the share of cyclists in each sample. The seed fixes the random choices, so you get the same five samples on every run.

Parameter and statistic

You met these words in the chapter on sampling. Let us use them again.

• A parameter is a number that describes the whole population. The true share $p = 0.35$ is a parameter. So is the mean travel time of all $1200$ students, written $\mu$.
• A statistic is a number worked out from a sample. The share of a sample, written $\hat p$ and read "p hat", is a statistic. So is the mean of a sample, written $\bar x$ and read "x bar".

A parameter is fixed, but we usually do not know it. A statistic is known once we have the sample, but it changes from sample to sample. This change is called sampling variability.

Means scatter too

The same happens with a mean. Suppose the travel times of the school have mean $\mu = 24$ minutes and standard deviation $10$ minutes. Each sample of $10$ students gives its own mean: perhaps $21.3$ minutes for one sample and $26.8$ minutes for the next.

The simulation draws twelve samples at a time. Each dot is the mean of one sample.

You should see the dots scatter on both sides of the red line. They land below and above it about equally often. When the samples are larger, the dots crowd closer to $24$ minutes. The grey band holds about $95$ of every $100$ dots in the long run. Course 2 of this chapter uses that band.

Chance scatter and bias

There are two kinds of error, and they behave differently.

1. Chance scatter. Random samples miss in both directions. Larger samples miss by less.
2. Bias. A method that leaves people out, such as asking only the students at the bicycle stand, misses in the same direction every time. A larger sample does not fix it.

A dart player can help you picture this. A player whose darts land all around the centre has scatter. A player whose darts all land in a tight group at the top left has bias.

Practise with new numbers

Common mistakes

• Thinking a sample that misses was done wrongly. A random sample misses by chance. That is normal, and we can measure how much.
• Mixing up $p$ and $\hat p$. The parameter $p$ belongs to the population. The statistic $\hat p$ belongs to one sample.
• Trusting a large biased sample. Size shrinks the scatter, not the bias.
• Expecting the next sample to correct the last one. Each random sample is new. Samples do not balance each other out on purpose.

Remember these

• A parameter describes the population, and a statistic describes a sample.
• Statistics such as $\hat p$ and $\bar x$ change from sample to sample. This is sampling variability.
• Random samples scatter on both sides of the truth. A biased method misses in one direction.

Check yourself

Try each one on your own first. If you are stuck, open a hint. Then check with Show the answer.

1. A farm has $3000$ eggs. A worker weighs $20$ of them, chosen at random, and finds a mean mass of $58$ g. Which number is the parameter, and which is the statistic?

2. In the school with true share $0.35$, a random sample of $40$ students holds $18$ cyclists. Find the sample share and how far it misses the true share.

3. Two classmates take fair random samples of the same size from the same population. Their sample means differ. Did one of them make a mistake?

4. Which scatter more: the means of samples of $5$ students, or the means of samples of $50$ students?

5. Ten groups each take a sample of $40$ students at the school gate, and every group gets a share of cyclists near $0.55$. The true share is $0.35$. Is this chance scatter or bias?

References

• OpenStax, Introductory Statistics, Chapter 1, section on sampling and variation in data, and Chapter 7 on the central limit theorem.