The margin of luck in a poll

What you will be able to do

• find the margin of luck of a fair poll with the quick rule $\dfrac{100}{\sqrt n}$;
• find the sample size that gives a chosen margin;
• read a poll result with its margin, and say what the margin leaves out.

52% plus or minus what?

A news report says, "A poll of 400 adults finds that 52% support the new road." Does a majority of the town support it?

From the first course you know that a random sample strays from the truth by luck. A careful report gives the size of that luck as a margin of error. Here we call it the margin of luck, because that is all it covers.

The quick rule

For a fair random poll of $n$ people, about 95 polls in 100 land within

of the true percentage. For $n = 400$, the margin is $\dfrac{100}{20} = 5$ percentage points. So the poll's 52% means "probably between 47% and 57%". A majority is not certain.

Where does the rule come from? In the first course the band of luck was $2\sqrt{\dfrac{p(100 - p)}{n}}$. Its top, $p(100 - p)$, is largest when $p = 50$, and then the band is $2 \times \dfrac{50}{\sqrt n} = \dfrac{100}{\sqrt n}$. So the quick rule is the widest the band can be. It is a little generous when the true percentage is far from 50. Chapter 6 of this book gives the exact formula and the meaning of "95 in 100".

How many people do we need?

Turn the rule round. We want the margin $m$, so $\sqrt n = \dfrac{100}{m}$, and

For a margin of 5 points, $n = 20^2 = 400$. For 2 points, $n = 50^2 = 2500$. For 1 point, $n = 100^2 = 10\,000$. Halving the margin needs four times the people, which is why most national polls stop at one or two thousand.

Notice what is missing: the size of the population. A random sample of 1000 has about the same margin in a town of $50\,000$ as in a country of millions. This holds as long as the sample is a small part of the population.

What the margin does not cover

• Bias. The margin covers luck only. A voluntary response poll of $100\,000$ has a tiny margin and may still be far from the truth.
• Small groups inside the poll. If 100 of the 400 people are under 25, the margin for that group is $\dfrac{100}{\sqrt{100}} = 10$ points.
• A lead between two choices. If 52% back one plan and 48% the other, the lead is 4 points. The lead has a margin of about twice the poll's margin, because when one share goes up, the other goes down.

Practise with new numbers

Try it in Python

The program runs 1000 polls of 400 people each, in a made-up town where 52% support a plan. It counts how many polls land within the margin of the truth.

Your turn.

The usual mistakes

• Reading a result as exact. "52%" from a poll of 400 means "probably between 47% and 57%".
• Thinking a bigger population needs a much bigger sample. The margin depends on $n$, not on the size of the population.
• Thinking the margin covers bias. It covers only the luck of a fair random sample.
• Doubling the sample to halve the margin. You need four times the sample.

Remember these

• Margin of luck $\approx \dfrac{100}{\sqrt n}$ percentage points, for about 95 polls in 100.
• To get a margin $m$, take $n = \left(\dfrac{100}{m}\right)^2$.
• Small groups inside a poll have larger margins. A lead has a margin of about twice the poll's margin.
• The margin is about luck only, never about bias.

Check yourself

Try each one on your own first. If you are stuck, open a hint. Then check with Show the answer.

1. Find the margin of luck for a fair poll of 2500 people.

2. A fair poll of 100 people finds 61% in favour. Between which two percentages does the truth probably lie?

3. How many people does a poll need for a margin of 4 percentage points?

4. Two random polls, one in a city of 5 million and one in a town of $20\,000$, each ask 1000 people. Which has the larger margin of luck?

5. A fair poll of 900 finds 49% for plan A and 51% for plan B. A reporter says, "Plan B wins." Is that fair?

References

• OpenStax, Introductory Statistics, Chapter 8, on confidence intervals for a population proportion, which gives the exact form of the margin of error.