To offer a beacon through the often muddy waters of interpreting evidence, The Abdul Latif Jameel Poverty Action Lab has released an informal guide describing factors that affect randomised evaluations’ statistical power, the sensitivity of an evaluation to detect any change brought about by the programme. Six Rules of Thumb for Determining Sample Size and Statistical Power describes how relationships between these factors affect a study’s design and results.
The main points outlined in the guide are:
- Larger sample sizes, which are the amount of subjects in a study, increase statistical power.
- If effect sizes are small, larger sample sizes can help achieve a given level of power.
- Evaluations of small programmes need larger sample sizes.
- If outcomes vary drastically among study subjects, a larger sample size is needed.
- Study subjects should be divided equally between experimental and control groups.
- In randomised evaluations, randomising in “clusters,” or groups, is less powerful than individual random assignment.
A companion piece on the dangers of performing under-powered evaluations can be found here.