🧪 This platform is in early beta. Features may change and you might encounter bugs. We appreciate your patience!
will_rogers_phenomenon
A statistical artifact where the average of every group improves when members are reclassified from one group to another, without any actual improvement in individual outcomes. Named after Will Rogers' joke: 'When the Okies left Oklahoma and moved to California, they raised the average intelligence in both states.'
Improved cancer diagnostic technology reclassifies patients from Stage I to Stage II. Stage I survival improves (the worst cases left), and Stage II survival also improves (the new additions are the mildest Stage II cases).
A school district reassigns its weakest students from 'advanced' classes to 'standard' classes. The average test score in the advanced group rises (the lowest performers left), and the average in the standard group also rises (the reassigned students outperform the existing standard group). Administrators proudly report that both programs improved, though no student learned more.
A financial advisor moves underperforming stocks from a 'high-growth' portfolio to a 'balanced' portfolio. The average return of the high-growth portfolio improves, and the average return of the balanced portfolio also improves because the moved stocks still beat that group's weakest holdings. The advisor claims both portfolios are now performing better, but total wealth is unchanged.
Binary (yes/no) questions an LLM must answer to identify this aspect:
Are subjects being classified into categories or stages?
Type: binaryHas the classification criteria changed, moving subjects from one category to another?
Type: binaryDoes this reclassification cause the average outcome in both categories to appear to improve, without any actual change in individual outcomes?
Type: binaryA statistical artifact where the average of every group improves when members are reclassified from one group to another, without any actual improvement in individual outcomes. Named after Will Rogers' joke: 'When the Okies left Oklahoma and moved to California, they raised the average intelligence in both states.'
When the worst members of a good group are moved to become the best members of a bad group, both group averages improve mathematically. No individual's outcome has changed.
Track individual outcomes over time rather than group averages. Be suspicious when diagnostic improvements coincide with apparent outcome improvements across all stages.
Cancer staging, education tracking, poverty statistics, and any system where improved classification reclassifies borderline cases.
Use these tools to detect, analyze, or train this aspect.