CI Misinterpretation — When Logic Wears a Disguise
A 95% confidence interval is widely misinterpreted as meaning there is a 95% probability that the true parameter lies within the calculated interval. The correct interpretation is procedural: if the study were repeated infinitely many times, 95% of the intervals constructed would contain the true parameter. For any specific interval, the true parameter is either inside or outside — a statement about procedure, not about any particular interval.
Also known as: CI misuse, Frequentist interval confusion
How It Works
The correct interpretation is procedural and abstract; the incorrect interpretation is intuitive and direct. Bayesian credible intervals do have the probabilistic interpretation, so the conceptual confusion is easy to make.
A Classic Example
A study reports that the treatment effect is 5.2 points (95% CI: 2.1, 8.3). A journalist writes 'there is a 95% chance the true effect is between 2.1 and 8.3 points.' This is incorrect — the 95% refers to the long-run performance of the procedure, not to this specific interval.
More Examples
A polling firm reports that a candidate's approval rating is 52% (95% CI: 49%, 55%). A news anchor tells viewers: 'We are 95% confident the candidate's true approval is somewhere in that range right now.' In fact, the interval is a property of the repeated sampling procedure — the true approval is a fixed value, and this particular interval either captures it or it doesn't.
A pharmaceutical company's press release states: 'There is a 95% probability that our drug reduces blood pressure by between 3 and 9 mmHg.' A statistician flags this as misleading — the 95% refers to the long-run frequency with which such intervals, constructed from repeated studies, would contain the true fixed effect, not to a probability about this specific interval.
Where You See This in the Wild
Surveys of scientists find that over 80% misinterpret confidence intervals, affecting peer review, public communication, and policy decisions.
How to Spot and Counter It
State CIs in procedural terms. Use Bayesian credible intervals when a probabilistic statement about a specific interval is required. Do not conflate CI width with posterior uncertainty.
The Takeaway
The CI Misinterpretation is one of those reasoning errors that sounds perfectly logical at first glance. That's what makes it dangerous — it wears the costume of valid reasoning while smuggling in a broken conclusion. The best defense? Slow down and ask: does this conclusion actually follow from these premises, or am I just connecting dots that happen to be near each other?
Next time someone presents you with an argument that "just makes sense," check the structure. The feeling of logic is not the same as logic itself.