🧪 This platform is in early beta. Features may change and you might encounter bugs. We appreciate your patience!
outcome_bias
The tendency to judge a decision by its eventual outcome rather than by the quality of the reasoning process at the time it was made. A good decision that leads to a bad outcome (due to chance) is judged harshly, while a bad decision that luckily turns out well is praised.
A surgeon who follows best practices but loses a patient is judged as incompetent, while a surgeon who takes unnecessary risks but gets lucky is praised as bold and skilled.
A general who made a strategically sound retreat to save troops is labeled a coward after the war is lost, while a general who made a reckless charge that happened to succeed is celebrated as a military genius.
A product manager who carefully researched and launched a feature that flopped due to an unforeseeable market shift is passed over for promotion, while a colleague whose hastily conceived feature accidentally went viral is fast-tracked to leadership.
Binary (yes/no) questions an LLM must answer to identify this aspect:
Is the quality of a decision being evaluated?
Type: binaryIs the evaluation based on the outcome of the decision rather than on the quality of the reasoning at the time it was made?
Type: binaryWould the evaluation change if the outcome had been different, even with identical reasoning?
Type: binaryThe tendency to judge a decision by its eventual outcome rather than by the quality of the reasoning process at the time it was made. A good decision that leads to a bad outcome (due to chance) is judged harshly, while a bad decision that luckily turns out well is praised.
Outcomes are concrete and salient, while the reasoning process is abstract and harder to evaluate. Hindsight makes it seem like the outcome was foreseeable.
Evaluate decisions based on the information and reasoning available at the time. Judge the process, not the result.
Performance reviews, sports coaching decisions, medical malpractice cases, and investment fund manager evaluations.
Use these tools to detect, analyze, or train this aspect.