The Base Rate Fallacy: "99% Accurate" Doesn't Mean What You Think

A Test That Lies to You

Imagine there's a deadly new disease. Only 1 in 1,000 people has it.

Scientists develop an amazing test for it. It's 99% accurate — meaning if you have the disease, the test says "positive" 99% of the time. And if you don't have it, the test says "negative" 99% of the time.

You take the test.

It comes back positive.

How worried should you be?

Most people hear "99% accurate test came back positive" and immediately think: I have a 99% chance of being sick.

The actual answer? You probably don't have the disease. Like, there's roughly a 9 in 10 chance you're fine.

How is that possible? Welcome to the base rate fallacy.

Let's Do the Math (Don't Panic)

Let's say we test 100,000 people.

Who actually has the disease?

1 in 1,000 people → that's 100 people who are sick.

True positives: The test catches 99% of sick people → 99 correct positive results

False negatives: The test misses 1% of sick people → 1 person is sick but tests negative (we'll ignore them)

Who doesn't have the disease?

That leaves 99,900 healthy people.

True negatives: The test correctly clears 99% of healthy people → 98,901 people get correct negatives

False positives: The test incorrectly flags 1% of healthy people → 999 healthy people test positive

Now count all the positive results:

• 99 real positives (actually sick)
• 999 false positives (healthy but flagged)
• Total positives: 1,098

Out of 1,098 people who tested positive... only 99 are actually sick.

That means if you test positive: your chance of actually being sick is 99 ÷ 1,098 = about 9%.

A 99% accurate test gives you a positive result that's wrong 91% of the time.

Your brain just broke a little, right? That's normal.

Why Does This Happen?

The test isn't lying. The math is working exactly as it should.

The problem is that we forgot about the base rate — how common the thing actually is in the first place.

When something is rare (like 1 in 1,000), even a tiny false positive rate gets multiplied across a massive healthy population. Those false alarms swamp the real ones.

It's like having a smoke detector so sensitive it goes off when someone makes toast. Most of the alarms it ever triggers are not fires. Not because the detector is bad — because actual fires are rare, and toast is common.

Real-Life: This Gets People Hurt

Drug testing at school/work

If drug use is rare in a group (say 1%), even a highly accurate test will produce more false positives than true positives. People lose jobs or face punishment for test results that are statistically likely to be wrong. Courts have grappled with this for decades.

Disease screening programs

Doctors know about base rates. That's why they don't screen the whole population for rare cancers — too many false positives lead to unnecessary, sometimes dangerous follow-up procedures. They target high-risk groups where the base rate is higher.

"I saw someone like him on the news, so..."

When people see a crime committed by someone of a particular group, they sometimes update their fear of everyone in that group. But if the base rate of people in that group who commit crimes is very low, the vivid example on the news doesn't actually change the real probability much. This is base rate neglect causing real-world harm.

AI content detection

"This tool detects AI-written text with 98% accuracy!" If only 5% of texts are actually AI-written, and the tool has a 2% false positive rate... lots of students get flagged for writing their own homework.

How to Not Get Fooled

Whenever someone tells you about a test, a detection system, or a probability:

Ask: "What's the base rate?"

• How common is the thing being tested for in this population?
• What's the false positive rate (not just the accuracy)?
• How many real cases vs. false alarms will there actually be?

A useful mental check: visualize the whole population being tested, not just the person in front of you.

100,000 people walk through the door. How many really have it? How many will the test flag? Of those flagged, how many actually have it?

This turns an abstract probability into something your brain can actually process.

The Challenge

Here's a scenario to work through:

A school uses a lie detector app to catch cheating (fictional, but go with it). It's advertised as 95% accurate. In a class of 200 students, maybe 10 students actually cheated.

Work out:

• How many true positives will the test produce?
• How many false positives (honest students flagged)?
• If a student gets flagged, what's the actual probability they cheated?

Then: Is this a fair system? Write 3 sentences on whether you'd accept this test as evidence of cheating.

"99% accurate" sounds like certainty. It's not. The number that actually matters is one most people never ask about.