Underpowered Study — When Numbers Lie
Has this ever happened to you? A study with 15 participants per group tests whether a new therapy reduces depression symptoms.
Also known as: low statistical power, small sample study, insufficient sample size
What's Actually Happening
An underpowered study has too few participants or observations to reliably detect an effect of the expected size. Statistical power is the probability that a study will detect a true effect when one exists. Studies with power below 80% (a common convention) are considered underpowered. Such studies produce unreliable results: significant findings are likely inflated in magnitude, and non-significant findings cannot be interpreted as evidence of no effect.
Sample size calculations are technical and rarely reported in the media or in press releases. Audiences assume that any published study is adequately sized, treating non-significant results as definitive null findings.
Real Talk: You See This Every Day
A study with 15 participants per group tests whether a new therapy reduces depression symptoms. The expected effect size requires 80 participants per group for 80% power. The study finds p = 0.08 and concludes 'no significant effect.' This does not mean the therapy does not work; the study simply lacked the sample to detect it.
Underpowered studies are common in neuroscience, pilot clinical trials, and social science experiments. Button et al. (2013) found the median statistical power of neuroscience studies was just 21%.
Your BS Detector
Check the reported sample size against the expected effect size. If the study is small and finds no effect, note that it may be underpowered. Look for power analyses in the methods section.
- ✓ Who collected this data, and why?
- ✓ Is the sample big enough and fair?
- ✓ Could there be another explanation?
The Challenge
Next time someone throws a statistic at you — in class, online, in the news — don't just accept it. Ask: what's missing from this picture?
Part of the TellDear Teen Book — criticalthinking.guide