Sums, differences and averages of random variables… | StemPy Lab
Free previewCourse: Standard
Sums, differences and averages of random variables
Two dice added and two dice taken away have the same spread, and an average of many readings is steadier than one. This lesson finds the mean and the variance of a sum, a difference and an average. It shows when the rule for variances breaks, and it combines independent normal variables to answer questions about parts that must fit.
This is the first lesson of Distributions in Depth: Sums, Approximations and Heavy Tails, free for everyone to read.
What you will be able to do
find the mean and the variance of a sum and a difference of two random variables;
say when the variances add and when they do not;
explain why the standard deviation of an average of n independent values is nσ;
find chances for a sum or a difference of independent normal variables.
Two dice, added and taken away
In the free course on expected value you found that the mean of a total is the total of the means. Now let us ask about the spread. Throw two fair dice, a red one and a blue one. Let X be the red number and Y the blue number.
One die has mean 3.5 and variance 1235≈2.917. We use the n formula here, because we list every equally likely outcome. The 36 pairs of numbers give these results.
One die, X: mean 3.5, variance 2.917, standard deviation 1.708.
Two dice added, X+Y: mean 7, variance 5.833, standard deviation 2.415.
Two dice taken away, X−Y: mean 0, variance 5.833, standard deviation 2.415.
One die doubled, 2X: mean 7, variance 11.667, standard deviation 3.416.
Look at the variances. The sum and the difference both have 635≈5.833, which is twice the variance of one die. Doubling one die gives four times the variance.
Three bar charts side by side with the same horizontal scale. The first shows one fair die, six equal bars at 1 to 6. The second shows the sum of two independent dice, a triangle of bars from 2 to 12 with its peak at 7. The third shows twice one die, six equal bars at 2, 4, 6, 8, 10 and 12, spread over the same range as the sum but with gaps between them.
The rules
For any two random variables, the mean of a sum and of a difference behave as you expect:
E(X+Y)=E(X)+E(Y),E(X−Y)=E(X)−E(Y).
For independent random variables the variances add, in both cases:
Var(X±Y)=Var(X)+Var(Y).
Why does a difference add the variances? Taking away Y is the same as adding −Y. The values of −Y are just as spread out as the values of Y. So the difference is as uncertain as the sum.
Two more rules from the free course still hold. Var(aX)=a2Var(X), and adding a constant does not change the variance. So Var(3X−2Y)=9Var(X)+4Var(Y) when X and Y are independent.
Standard deviations never add. Always add the variances first, then take the square root.
When the rule for variances breaks
Take one die. Let X be the top face and Y the bottom face. Opposite faces of a die add to 7, so Y=7−X. Each of X and Y has variance 2.917. Yet X+Y=7 every time, so its variance is 0.
The rule failed because X and Y are not independent. When X is high, Y is low, and the two cancel. Two variables that move together have the opposite effect: they make a sum more spread out.
The same idea explains the last row of the table. 2X is X+X, a variable added to itself. The two copies always move together, so the variance is 4×2.917, not 2×2.917. Two independent dice, X+Y, give only twice the variance.
Why an average is steadier
Measure the same length n times. Each reading has mean μ and standard deviation σ, and the readings are independent. The total has variance nσ2. The average is the total divided by n, so its variance is n2nσ2=nσ2. That gives
E(Xˉ)=μ,sd(Xˉ)=nσ.
The average of 25 dice has standard deviation 51.708≈0.342. Four times as many readings halve the spread of the average. In the chapter on learning from samples you watch this happen in a simulation. Here you see where the n comes from.
Combining normal variables
A useful fact goes with these rules. A sum or a difference of independent normal variables is normal too. So once you know its mean and its variance, you can find any chance from a z-score.
A made-up factory makes lids and jars. The inside width L of a lid is normal with mean 80.4mm and standard deviation 0.3mm. The outside width J of a jar is normal with mean 80.0mm and standard deviation 0.4mm. A lid picked at random is put on a jar picked at random. Does it fit?
Name the gap. Let D=L−J. The lid fits when D>0mm.
Its mean.E(D)=80.4mm−80.0mm=0.4mm.
Its spread.Var(D)=(0.3mm)2+(0.4mm)2=0.25mm2, so sd(D)=0.5mm.
The chance.P(D>0)=P(Z>0.50−0.4)=P(Z>−0.8)=0.7881.
So about 79% of random pairs fit. If a gap above 1.0mm makes the lid too loose, a good fit is 0<D<1. Its chance is Φ(1.2)−Φ(−0.8)=0.6731.
A normal curve for the gap between a made-up lid and a made-up jar, lid width minus jar width, in millimetres from minus 1.5 to 2.5. The curve has its peak at 0.4 mm. A dashed vertical line marks a gap of 0 mm, and the area under the curve to the right of that line is shaded to show the lids that fit.
Predict first. With the sliders at the start, the share that fits is 0.7881. Now make the lid machine exact, with σL=0.1mm. Does the share rise a lot or a little?
A worked example
Mina takes a bus and then walks. The bus ride is normal with mean 24 minutes and standard deviation 3.0 minutes. The walk is normal with mean 9 minutes and standard deviation 1.6 minutes, independently. She leaves 40 minutes before school starts. What is the chance that she is late?
The total.T = ride + walk has mean 24+9=33 minutes.
The spread.Var(T)=3.02+1.62=11.56min2, so sd(T)=3.4 minutes.
The z-score.z=3.440−33≈2.059.
The chance.P(T>40)=1−Φ(2.059)≈0.0198, about 2% of days.
Check. If you add the standard deviations, you get 4.6 minutes and a chance near 6%, three times too large.
Your turn
Try it in Python
The program lists all 36 pairs of two dice with exact fractions. It finds the mean and the variance of the sum, the difference, and the top face plus the bottom face. Then it simulates 2000 averages of 25 dice with a fixed seed.
The usual mistakes
Adding standard deviations. Add the variances, then take the square root.
Taking away variances for a difference.Var(X−Y) is Var(X)+Var(Y), never the difference.
Treating 2X as X1+X2. One value doubled has four times the variance. Two independent values added have twice the variance.
Forgetting independence. The variance rule needs independent parts. The mean rule does not.
Remember these
E(X±Y)=E(X)±E(Y), always.
Var(X±Y)=Var(X)+Var(Y) for independent X and Y.
An average of n independent values has standard deviation nσ.
A sum or a difference of independent normal variables is normal.
Check yourself
Try each one on your own first. If you are stuck, open a hint. Then check with Show the answer.
X and Y are independent, with Var(X)=4 and Var(Y)=9. Find the standard deviation of X−Y.
A student says that the top face plus the bottom face of one die must have variance 635, like two dice. What is wrong?
One reading of a made-up length has a standard deviation of 2.4cm. What is the standard deviation of the average of 36 independent readings?
A made-up task has two stages. Stage one is normal with mean 30 minutes and standard deviation 2 minutes. Stage two is normal with mean 25 minutes and standard deviation 1.5 minutes, independently. Find the chance that the whole task takes more than 60 minutes.
E(X)=5, Var(X)=1, E(Y)=2 and Var(Y)=4, with X and Y independent. Find the mean and the standard deviation of 3X−2Y.
References
D. S. Moore, G. P. McCabe and B. A. Craig, Introduction to the Practice of Statistics, Chapter 4 (means and variances of random variables).
OpenStax, Introductory Statistics, Chapter 7 (the central limit theorem and the spread of a sample mean).
Liked this lesson? There is more
Distributions in Depth: Sums, Approximations and Heavy Tails is a Standard course. It adds 2 more lessons, practice and tests on top of the free chapter.
12Problems2Python Labs1Question Set1Mock Test
The normal curve as a stand-in for a count: A binomial count with many trials looks like a bell, so the normal curve can stand in for it. This lesson makes that stand-in exact enough to use, with the continuity correction, for a range, a single value and a proportion. It then shows where the stand-in fails: when the count is skewed, and far out in the tails.
Waiting times, rare events and heavy tails: Not every distribution is a bell. This lesson meets two models that the binomial turns into: the geometric model for a wait, and the Poisson model for rare events. It then shows skewed and heavy-tailed models that break the 95 per cent rule, and a bound that holds for every distribution.