Underpowered Study — When Logic Wears a Disguise

An underpowered study has too few participants or observations to reliably detect an effect of the expected size. Statistical power is the probability that a study will detect a true effect when one exists. Studies with power below 80% (a common convention) are considered underpowered. Such studies produce unreliable results: significant findings are likely inflated in magnitude, and non-significant findings cannot be interpreted as evidence of no effect.

Also known as: low statistical power, small sample study, insufficient sample size

How It Works

Sample size calculations are technical and rarely reported in the media or in press releases. Audiences assume that any published study is adequately sized, treating non-significant results as definitive null findings.

A Classic Example

A study with 15 participants per group tests whether a new therapy reduces depression symptoms. The expected effect size requires 80 participants per group for 80% power. The study finds p = 0.08 and concludes 'no significant effect.' This does not mean the therapy does not work; the study simply lacked the sample to detect it.

More Examples

A startup tests its mindfulness app on 12 employees to see if it reduces workplace stress. The study finds a small improvement that doesn't reach statistical significance (p = 0.11) and concludes the app 'shows no effect,' when in reality the sample was far too small to detect a plausible benefit.

Researchers investigate whether a rare genetic variant is associated with a neurological condition by recruiting 20 affected individuals and 20 controls. They find no significant association and publish a null result, but the study had only 15% power to detect the expected effect — the absence of evidence is mistaken for evidence of absence.

Where You See This in the Wild

Underpowered studies are common in neuroscience, pilot clinical trials, and social science experiments. Button et al. (2013) found the median statistical power of neuroscience studies was just 21%.

How to Spot and Counter It

Check the reported sample size against the expected effect size. If the study is small and finds no effect, note that it may be underpowered. Look for power analyses in the methods section.

The Takeaway

The Underpowered Study is one of those reasoning errors that sounds perfectly logical at first glance. That's what makes it dangerous — it wears the costume of valid reasoning while smuggling in a broken conclusion. The best defense? Slow down and ask: does this conclusion actually follow from these premises, or am I just connecting dots that happen to be near each other?

Next time someone presents you with an argument that "just makes sense," check the structure. The feeling of logic is not the same as logic itself.