Surveys, Experiments and Observational Studies: Quick Revision

A brief note on Surveys, Experiments and Observational Studies. Read it before a test. Try each question at the end before you open its answer.

Three kinds of study

• Sample survey: ask a sample questions.
• Observational study: record what people already do. It shows a link, not a cause.
• Experiment: impose a treatment. A well-designed one can show a cause.
• Explanatory variable: the possible cause. Response variable: the outcome.

A good survey question

• One idea, neutral words, a clear time and unit.
• Answer choices that cover every case and do not overlap.
• A pilot on a few people first.

A fair experiment

• Control group: no treatment, or the usual one.
• Random assignment: makes the groups alike on average.
• Placebo: a dummy treatment, for the placebo effect.
• Blinding: single (subjects) or double (subjects and researchers).
• Blocks: alike subjects together, chance inside each block.
• Effect: treatment group minus control group.

Confounding and risk

• Confounder: linked to the explanatory variable and affects the response.
• A link can also come from reverse cause or luck.
• Relative risk: the change as a percentage of the old risk. Absolute risk: the change in people per 1000.

A worked example

A risk rises from 10 to 12 people in every 1000.

1. Relative rise. $\dfrac{12 - 10}{10} \times 100 = 20\%$.
2. Absolute rise. $12 - 10 = 2$ people in every 1000.
3. In a town of $30\,000$. $2 \times 30 = 60$ extra cases.

Practice with new numbers

A program to try

The program works out the three answers of the worked example.

Mistakes to avoid

• A cause from a link. Look for a confounder or a reverse cause.
• No control group. The effect is a difference between groups.
• Choosing the groups by hand. Use a chance device.

Check yourself

1. A study gives half of 80 mice, chosen by a shuffle, a new food. Is it an experiment?

2. What does a placebo guard against?

3. Cities with more libraries have more crime. Name a confounder.

4. A risk rises from 2 to 3 in 1000. Give the relative rise.

Sources

• OpenStax, Introductory Statistics, Chapter 1, section on experimental design and ethics.