🧪 This platform is in early beta. Features may change and you might encounter bugs. We appreciate your patience!
prosecutors_fallacy
The prosecutor's fallacy involves confusing the probability of the evidence given innocence — P(evidence | innocent) — with the probability of innocence given the evidence — P(innocent | evidence). A DNA match with a 1-in-a-million coincidence probability does not mean there is only a 1-in-a-million chance the defendant is innocent, because it ignores how many people were in the suspect pool.
A forensic expert testifies that there is a 1-in-10-million chance that an innocent person would share the defendant's DNA profile. The prosecutor argues this means there is a 1-in-10-million chance the defendant is innocent. In a city of 3 million people, approximately 0.3 people would match by chance — making the coincidence less improbable than stated.
A statistician testifies that only 1 in 50,000 people have the same rare shoe size and tread pattern found at a crime scene. The prosecutor tells the jury: 'That means there's a 1 in 50,000 chance the defendant is innocent.' But in a city of 500,000 people, roughly 10 people share that profile — the evidence alone says nothing close to that about guilt.
A cybersecurity analyst finds that a server access pattern matches a known hacker's behavior with a probability of 0.002% for any random innocent user. The company's legal team argues this means there is a 0.002% chance the accused employee is innocent. They ignore that across millions of users worldwide, hundreds could produce the same pattern by chance.
Binary (yes/no) questions an LLM must answer to identify this aspect:
Is a probability statement about evidence being confused with a probability statement about guilt or innocence?
Type: binaryIs P(evidence | hypothesis) being treated as equal to P(hypothesis | evidence)?
Type: binaryDoes the argument ignore the base rate probability of the hypothesis?
Type: binaryCould Bayes' theorem be applied to reveal how the two probabilities differ?
Type: binaryThe prosecutor's fallacy involves confusing the probability of the evidence given innocence — P(evidence | innocent) — with the probability of innocence given the evidence — P(innocent | evidence). A DNA match with a 1-in-a-million coincidence probability does not mean there is only a 1-in-a-million chance the defendant is innocent, because it ignores how many people were in the suspect pool.
P(A|B) and P(B|A) can differ dramatically. This transposition is cognitively natural and affects even experts. Small conditional probabilities feel like strong proof regardless of the base rate.
Apply Bayes' theorem explicitly. Ask: given the number of people who could plausibly be suspects, how many would also match the evidence by chance? The answer gives the true posterior probability.
Multiple wrongful convictions have been attributed to the prosecutor's fallacy. The Meadow's Rule case (Sally Clark) is a textbook example where a 1-in-73-million probability was misrepresented as the probability of innocence.
Use these tools to detect, analyze, or train this aspect.