Undercoverage Bias — When Logic Wears a Disguise
Undercoverage bias occurs when some members of the target population have zero or near-zero probability of being included in the sample because the sampling frame does not cover them. Classic examples include telephone surveys that miss people without phones, or online surveys that miss people without internet access. The excluded groups often differ systematically on the very variables being measured.
Also known as: Frame bias, Coverage error
How It Works
Sampling frames are constructed for convenience and availability, not completeness. Researchers often underestimate how excluded groups differ from included ones.
A Classic Example
A 1936 Literary Digest poll predicted a landslide victory for Alf Landon based on 2.4 million responses, but only surveyed car owners and telephone subscribers, systematically excluding poorer voters who overwhelmingly supported Roosevelt.
More Examples
A city conducts a resident satisfaction survey by emailing registered voters. Renters who move frequently, undocumented residents, and people without stable internet access are almost entirely excluded, making satisfaction scores appear higher than they would if the full residential population were captured.
A national health survey is conducted via landline telephone calls during weekday business hours. Working-age adults, people with only mobile phones, and shift workers are systematically underrepresented, causing the survey to overestimate retirement-age demographics and underestimate health issues common in younger working populations.
Where You See This in the Wild
Surveys of internet users, smartphone owners, or social media participants systematically exclude older, poorer, or less tech-savvy populations.
How to Spot and Counter It
Explicitly define the target population and evaluate how well the sampling frame covers it. Supplement with alternative data sources for underrepresented groups. Use weighting to adjust for known coverage gaps.
The Takeaway
The Undercoverage Bias is one of those reasoning errors that sounds perfectly logical at first glance. That's what makes it dangerous — it wears the costume of valid reasoning while smuggling in a broken conclusion. The best defense? Slow down and ask: does this conclusion actually follow from these premises, or am I just connecting dots that happen to be near each other?
Next time someone presents you with an argument that "just makes sense," check the structure. The feeling of logic is not the same as logic itself.