Insensitivity to Sample Size — The Trick You Don't See Coming

Also known as: Sample size neglect

🔥 Hook

A hospital administrator compares two surgical techniques.

🧠 What's Actually Happening?

The tendency to underappreciate the role of sample size in the reliability of statistical results. People expect small samples to be just as representative as large ones, leading them to draw strong conclusions from insufficient data. This violates the law of large numbers, which states that larger samples are more reliable.

Here's the sneaky part: People assess probability using the representativeness heuristic, judging by similarity to the expected outcome. Since both small and large samples can produce the same percentage, people treat them as equally informative.

📱 Real-Life Scroll

Online: A hospital administrator compares two surgical techniques. Technique A succeeded in 8 out of 10 cases (80%) and Technique B in 75 out of 100 cases (75%). The administrator chooses Technique A, ignoring that the small sample of 10 is far less reliable and the 80% could easily be due to chance.

Another one

A travel blogger reads that 4 out of 5 guests at a boutique hotel left five-star reviews and enthusiastically recommends it to her followers. She overlooks that only five guests had reviewed the hotel, making the 80% rating far less reliable than a competing hotel's 76% rating from 2,000 reviews.

IRL: This bias affects medical research interpretation, product reviews (giving weight to a few reviews), small-school effects in education policy, and A/B testing in technology companies.

🔍 How to Spot It

Always consider sample size when evaluating statistics. Be skeptical of conclusions drawn from small samples and look for larger datasets before making decisions.

✓ Is my brain shortcutting right now?
✓ What would change my mind? If nothing — red flag.
✓ Who benefits from me not noticing this?

🎯 Your Challenge

Spot one example this week. Write it down. Name it. That's how you level up.

Part of the TellDear Teen Book — criticalthinking.guide