Convenience Sampling Bias — When Logic Wears a Disguise
Convenience sampling bias arises when researchers sample whoever is most easily accessible rather than drawing a representative sample from the target population. The most pervasive example is psychology's over-reliance on WEIRD (Western, Educated, Industrialized, Rich, Democratic) samples, particularly American undergraduates, for conclusions claimed to be universally human.
Also known as: WEIRD sample bias, Availability sampling bias
How It Works
Undergraduate samples are cheap, available, and compliant. Researchers rationalize that cognitive and social processes are universal enough to generalize, but this assumption is rarely tested.
A Classic Example
Most foundational psychology studies on cooperation, fairness, and decision-making used US undergraduate students as subjects. When replicated with non-WEIRD populations, many results failed to hold, revealing the sample-specific nature of the original findings.
More Examples
A nutritionist studies the eating habits of 'average adults' by surveying patrons at a health food café near her university. The sample is overwhelmingly health-conscious and educated, producing dietary patterns far removed from those of the general population.
A political scientist tests a new survey instrument about civic engagement by distributing it to attendees at a local town hall meeting. Since town hall attendees are already unusually politically active, the instrument appears to detect high engagement everywhere it is later deployed, masking its inability to capture apathetic or disengaged citizens.
Where You See This in the Wild
A 2010 meta-analysis found that US undergraduates are among the most psychologically unusual populations on Earth on dimensions like individualism and visual perception, yet represent the bulk of the psychology literature.
How to Spot and Counter It
Ask who was sampled and why. Evaluate whether the population of interest matches the sample. Look for cross-cultural replications before accepting universal claims.
The Takeaway
The Convenience Sampling Bias is one of those reasoning errors that sounds perfectly logical at first glance. That's what makes it dangerous — it wears the costume of valid reasoning while smuggling in a broken conclusion. The best defense? Slow down and ask: does this conclusion actually follow from these premises, or am I just connecting dots that happen to be near each other?
Next time someone presents you with an argument that "just makes sense," check the structure. The feeling of logic is not the same as logic itself.