The law of large numbers by simulation

What you will be able to do

• describe the law of large numbers and see it in a simulation;
• estimate how far a share of successes can be from the true probability after $n$ tries;
• explain why the count of heads can drift away from half while the share gets closer.

A coin on a computer

A fair coin has $P(\text{heads}) = 0.5$. Does that mean $5$ heads in every $10$ tosses? Try it. The program below tosses a computer coin and stops to look after $10$, $100$, $1000$, $10000$ and $100000$ tosses.

The law of large numbers

The picture shows the same idea for four runs of $2000$ tosses each. Every run starts wildly and then settles. This is the law of large numbers:

As the number of tries grows, the relative frequency of an event gets closer and closer to its probability.

So a relative frequency from a long record is a good estimate of a probability. That is why insurance companies, weather services and factories keep long records. The Python course on simulation and randomness goes further with random numbers.

How close is close?

Nobody can say exactly where the share will land. But there is a band that holds it in about $95$ runs out of $100$. For an event of probability $p$ tried $n$ times, the share lands within

of $p$. For a fair coin, $p(1-p) = 0.25$, so the half-width is $\dfrac{1}{\sqrt{n}}$. After $100$ tosses that is $0.1$, so the share of heads is almost always between $0.4$ and $0.6$. After $10000$ tosses it is $0.01$. To make the band ten times narrower, you need a hundred times more tosses. You will meet this band again in the chapter on learning from samples.

What the law does not say

The law is about the share, not the count. It does not say that heads and tails "even out" in number. In the program, the count of heads drifted $40$ above half, and the share was still close to $0.5$. A coin has no memory, and it never "makes up" for an early run of tails. Later tosses simply swamp the early ones. The course on independence comes back to this in the lesson on streaks.

The law also needs a fixed probability. If a machine gets worse every week, its long record mixes old and new chances.

Your turn

The usual mistakes

• Expecting exact halves. A fair coin in $100$ tosses gives exactly $50$ heads only about $8\%$ of the time.
• Thinking the count evens out. The share settles. The difference between heads and tails usually grows.
• Trusting a short run. Ten tries can give a share of $0.2$ or $0.8$ for a fair coin.
• Forgetting the square root. Four times as many tries make the band only half as wide.

Remember these

• Law of large numbers: in a long run, the relative frequency gets close to the probability.
• In about $95$ runs out of $100$, the share is within $2\sqrt{p(1-p)/n}$ of $p$. For a fair coin this is $\dfrac{1}{\sqrt{n}}$.
• The share settles. The count of successes does not have to come back to $np$.

Check yourself

Try each one on your own first. If you are stuck, open a hint. Then check with Show the answer.

1. A fair coin is tossed $400$ times. Between which two shares does the share of heads land in about $95$ runs out of $100$?

2. Asha tosses a coin $10$ times and gets $7$ heads. Ben tosses it $1000$ times and gets $520$ heads. Whose result gives a better estimate of $P(\text{heads})$?

3. After $50$ tosses Leo has $30$ heads. He says, "Tails must come up more often now, to even things out." Is he right?

4. How many tosses of a fair coin make the band for the share of heads $0.02$ on each side?

5. Why does the law of large numbers not help with a machine that wears out a little each week?

References

• OpenStax, Introductory Statistics, Chapter 3, the section on terminology and the law of large numbers.