Resampling, Power and Honest p-values: A Simulation… | StemPy Lab
Standard content
“Resampling, Power and Honest p-values: A Simulation Workshop” is available to Standard subscribers.
You have one sample of nine commute times and no formula for the error of its median. Draw new samples from the sample itself and watch the median move. Here you build the bootstrap and see where it breaks, test pairs and correlations by shuffling, and plan a study by its power. You also measure how often a 95% interval really catches the truth, and why peeking at the data makes false alarms common. The level is that of a first university course in statistics, A-Level Further Maths statistics and AP Statistics, with a timed paper.
3Lessons12Problems2Python Labs1Question Set1Mock Test
The bootstrap: resampling one sample: You have one sample and no formula for the uncertainty of its median. The bootstrap draws new samples from the sample itself and watches the median move. This lesson runs the bootstrap for a median and a trimmed mean, works out a small bootstrap exactly by counting, and shows where the method breaks.
Shuffling tests for pairs and for correlation: In the free course you shuffled two groups and compared their means. Shuffling can test much more. This lesson chooses the statistic with care, tests paired data by flipping signs, and tests a correlation by shuffling one variable. Each test is done by simulation and, for small data sets, by counting every case.
Power, coverage and stopping rules: A study that is too small will often miss a real effect. An interval used outside its conditions catches the truth less often than it promises. A researcher who peeks at the data and stops early makes false alarms far more likely. This lesson measures all three by simulation and by exact counting, and ends with honest reporting.
The chapter itself stays free
This is an extra course for Learning from Samples. Its main courses are open to everyone: