Apps

🧪 This platform is in early beta. Features may change and you might encounter bugs. We appreciate your patience!

Methodology

How we built a machine-readable taxonomy of human reasoning patterns.

1. The Six-Dimensional Model

Traditional fallacy taxonomies are flat lists organized by a single criterion (e.g., formal vs. informal). This limits their analytical power because real-world flawed reasoning rarely fits a single category. A conspiratorial argument might simultaneously employ a logical fallacy (false cause), exploit a cognitive bias (confirmation bias), use a propaganda technique (scapegoating), and commit a statistical error (cherry picking).

TellDear addresses this by organizing reasoning aspects along six orthogonal dimensions, each capturing a distinct facet of how reasoning can go wrong — or go right:

D1: Logical Fallacies

Structural and content errors in deductive and inductive reasoning. Includes formal fallacies (invalid logical forms like affirming the consequent) and informal fallacies (relevance, ambiguity, and presumption errors like ad hominem or straw man). Based on Aristotle's Sophistical Refutations, Hamblin's taxonomy, and modern critical thinking literature.

D2: Manipulation & Propaganda

Rhetorical and psychological techniques designed to persuade through deception rather than rational argument. Covers classic propaganda (IPA's seven techniques), digital-age disinformation tactics (firehose of falsehood, astroturfing), and interpersonal manipulation strategies (DARVO, gaslighting). Sources include RAND Corporation, EU DisinfoLab, and the Institute for Propaganda Analysis.

D3: Cognitive Biases

Systematic psychological deviations from rational judgment, organized by domain: decision-making biases (loss aversion, sunk cost), social biases (fundamental attribution error, in-group bias), memory biases (peak-end rule, illusory truth), attention biases (frequency illusion, salience bias), and probability biases (gambler's fallacy, base rate neglect). Based on Kahneman & Tversky's heuristics-and-biases program.

D4: Statistical Errors

Errors in data analysis, interpretation, and presentation. Covers research methodology pitfalls (p-hacking, publication bias, underpowered studies), data visualization deceptions (truncated axes, misleading aggregation), and probability reasoning errors (Simpson's paradox, Berkson's paradox). Grounded in Ioannidis (2005), Gigerenzer's risk literacy research, and ASA guidelines on p-values.

D5: Argumentation Schemes

Normative templates for rational argumentation from Walton, Reed & Macagno's compendium of 60+ schemes. Includes argument from expert opinion, argument from analogy, argument from consequences, practical reasoning, and others. Each scheme has critical questions that must be answered for the argument to succeed — failure to answer constitutes a fallacious use.

D6: Discourse Mechanics

Meta-level patterns in how discussions and debates are conducted. Covers bad-faith tactics (goalpost moving, tone policing, sealioning), digital discourse phenomena (concern trolling, just asking questions), and structural argument patterns (motte and bailey, Kafka trap). These are not flaws in individual arguments but in the process of argumentation itself.

2. Knowledge Graph Design

The TellDear knowledge graph is a directed labeled graph G = (V, E, L) where vertices represent reasoning concepts and edges represent typed relations between them. The graph serves as the terminological box (TBox) of a lightweight ontology.

Node Types

ClassCountDescription
dimension6Top-level organizational axes
categoryvariesSub-groupings within dimensions
formal_fallacy27Structurally invalid argument forms
informal_fallacy66Content-based reasoning errors
propaganda50Persuasion through deception
bias109Systematic cognitive deviations
stat_error79Data analysis errors
scheme26Argumentation templates
discourse94Discussion-level patterns

Relation Types

Edges in the graph are typed with one of six relation labels:

  • is_a — Taxonomic hierarchy (e.g., ad_hominem is_a informal_fallacy)
  • sub_type_of — Specialization within a category
  • triggers — Causal enablement (e.g., confirmation_bias triggers cherry_picking)
  • correlates_with — Cross-dimensional association
  • pipeline_next — Sequential processing dependency
  • tool_supports — Tooling for detection or analysis

3. Atomic Instruction Dataset (AID)

The Atomic Instruction Dataset is the operational core of TellDear. For each reasoning aspect, AID provides a sequence of 2–4 binary verification steps — yes/no questions that can be answered independently to determine whether the aspect is present in a given text.

This approach is inspired by the Atomic Skills methodology in NLP evaluation, where complex capabilities are decomposed into minimal, independently testable units. The binary constraint ensures that each step has a clear, unambiguous answer, reducing inter-annotator disagreement and enabling reliable automated assessment.

Example: Ad Hominem

Verification Steps:
Step 1: Does the argument attack a person rather than their claim?
Step 2: Is the personal attribute relevant to the claim's validity?
Step 3: Does the argument conclude the claim is false based on the person?
Detection: If steps 1 and 3 = YES and step 2 = NO → ad hominem detected.

Coverage Statistics

535
Aspects with AID
1838
Total Steps
3.4
Avg Steps/Aspect

4. Binary Verification Protocol

The TellDear analyzer implements a systematic verification protocol for each aspect:

  1. 1
    Text Segmentation — The input text is decomposed into Elementary Discourse Units (EDUs), each classified as claim, premise, rebuttal, or statement.
  2. 2
    Structural Mapping — Relations between EDUs are identified: which premises support which claims, which statements rebut which arguments.
  3. 3
    AID Scanning — For each of the 535 aspects, the verification steps are evaluated against the text. Each step produces a binary answer (yes/no/uncertain).
  4. 4
    Confidence Scoring — The proportion of "yes" answers determines the detection confidence: 100% = all steps confirmed, 50% = threshold for detection.
  5. 5
    Severity Assessment — The total number and confidence of detections determines an overall severity rating: clean, low, medium, high, or critical.

The protocol supports two execution modes: heuristic (client-side keyword matching for instant results) and deep analysis (LLM-powered evaluation for nuanced semantic understanding).

5. Cross-Dimensional Relations

A key innovation of TellDear is the systematic mapping of cross-dimensional relationships. These lateral connections reveal how reasoning flaws interact and reinforce each other:

Bias → Fallacy Triggers

Cognitive biases create psychological conditions that make certain fallacies more likely. For example, confirmation bias triggers cherry picking — a person who seeks confirming evidence will naturally select only supporting data points.

Fallacy → Propaganda Correlation

Many propaganda techniques are formalized versions of informal fallacies deployed at scale. Appeal to fear is both a logical fallacy and a propaganda technique; the distinction lies in whether it is an individual reasoning error or a deliberate persuasion strategy.

Statistical Error → Bias Reinforcement

Statistical errors can both result from and reinforce cognitive biases. Survivorship bias leads to sampling bias in data collection, which produces misleading results that further entrench the original bias.

These relationships are formally represented as triggers and correlates_with edges in the knowledge graph, enabling automated chain-of-reasoning analysis that goes beyond detecting individual flaws.

6. Sources & References

Core Books & Foundational Texts

  • Walton, D., Reed, C., & Macagno, F. (2008). Argumentation Schemes. Cambridge University Press. — Compendium of 60+ argumentation schemes with critical questions.
  • Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux. — Foundational work on dual-process theory and cognitive biases.
  • Toulmin, S. E. (1958). The Uses of Argument. Cambridge University Press. — Introduced the Toulmin model of argumentation (claim, data, warrant, backing, qualifier, rebuttal).
  • Bennett, B. (2012). Logically Fallacious: The Ultimate Collection of Over 300 Logical Fallacies. eBookIt.com. — Comprehensive catalog of fallacies with examples.
  • Damer, T. E. (2008). Attacking Faulty Reasoning: A Practical Guide to Fallacy-Free Reasoning. 7th ed. Wadsworth. — Systematic framework for identifying and countering reasoning errors.
  • Tindale, C. W. (2007). Fallacies and Argument Appraisal. Cambridge University Press. — Modern analytical treatment of fallacy theory.
  • Govier, T. (2013). A Practical Study of Argument. 7th ed. Cengage Learning. — Standard textbook on argument analysis and evaluation.
  • Gilovich, T., Griffin, D., & Kahneman, D. (2002). Heuristics and Biases: The Psychology of Intuitive Judgment. Cambridge University Press. — Collected research on systematic cognitive errors.
  • Hamblin, C. L. (1970). Fallacies. Methuen. — Seminal historical and analytical survey of fallacy theory from Aristotle onward.
  • Baron, J. (2007). Thinking and Deciding. 4th ed. Cambridge University Press. — Comprehensive treatment of rational decision-making and its failures.
  • van Eemeren, F. H. & Grootendorst, R. (2004). A Systematic Theory of Argumentation: The Pragma-Dialectical Approach. Cambridge University Press. — The pragma-dialectical framework for analyzing argumentative discourse.
  • van Eemeren, F. H. & Grootendorst, R. (1992). Argumentation, Communication and Fallacies: A Pragma-Dialectical Perspective. Lawrence Erlbaum. — Fallacies as violations of discussion rules.
  • Tversky, A. & Kahneman, D. (1974). Judgment under Uncertainty: Heuristics and Biases. Science, 185(4157), 1124–1131. — The foundational paper on cognitive heuristics.

Key Datasets & Corpora

  • LOGIC & LOGICCLIMATE — Datasets containing educational examples of logical fallacies and climate change news articles annotated for reasoning errors. Jin et al. (2022).
  • COCOLOFA — Large-scale dataset of 7,706 news comments annotated for common logical fallacies. Enables training and evaluation of fallacy detection models at scale.
  • SAFEPERSUASION — Dataset of 1,887 social media comments categorizing rational persuasion versus manipulative tactics. Bridges the gap between fallacy detection and persuasion analysis.
  • Propaganda Techniques Corpus (PTC) & SemEval 2023 Task 3 — Datasets for fine-grained, span-level propaganda and persuasion detection across multiple languages. Da San Martino et al.
  • MissciPlus — Dataset grounded in real biomedical publication passages, focusing on fallacies that misrepresent scientific evidence. Critical for health misinformation detection.
  • NLAS (Natural Language Argumentation Scheme) — The largest publicly available corpus of natural language argumentation schemes, covering 20 Walton schemes across various topics.
  • QT-SCHEMES — 441 arguments from political debates mapped to Walton's argumentation schemes. Enables scheme-level argument mining in political discourse.
  • RuozhiBench — Dataset of 677 curated queries for evaluating LLMs on deceptive inputs, subtle logical traps, and misleading premises.
  • ABBA — Annotated dataset covering bias dimensions including queerphobia, islamophobia, and other forms of discriminatory reasoning in text.
  • ARIES & ArgAnalysis35k — Benchmarks for argument relation identification and large-scale argument quality analysis.
  • AIFdb / The Argument Web — The largest publicly available infrastructure for storing, analyzing, and sharing interconnected arguments and debates. Reed & Rowe.

Academic Papers & Methodological Frameworks

  • Jin, Z. et al. (2022). Logical Fallacy Detection. Findings of EMNLP 2022. — Introduces the LOGIC dataset and foundational AI taxonomy for fallacy detection.
  • Mouchel, L. et al. (2025). A Logical Fallacy-Informed Framework for Argument Generation. NAACL 2025. — Using fallacy awareness to improve argument generation quality.
  • Alhindi, T. et al. Follow My Lead: Logical Fallacy Classification with Knowledge-Augmented LLMs. — Knowledge-augmented approaches to automated fallacy classification.
  • Walton, D. Argument Mining by Applying Argumentation Schemes. — Applying Walton's scheme taxonomy to computational argument mining.
  • Ioannidis, J. P. A. (2005). Why Most Published Research Findings Are False. PLoS Medicine, 2(8). — Foundational analysis of statistical errors in scientific research.
  • Head, M. L. et al. (2015). The Extent and Consequences of P-Hacking in Science. PLoS Biology, 13(3). — Text-mining study of statistical manipulation in published research.
  • Paul, C. & Matthews, M. (2016). The Russian "Firehose of Falsehood" Propaganda Model. RAND Corporation, PE-198-OSD. — Analysis of high-volume, multi-channel propaganda strategy.
  • Beukeboom, C. J. & Burgers, C. (2017). Automating the Detection of Linguistic Intergroup Bias Through Computerized Language Analysis. — Computational approaches to detecting bias in language.
  • Stammbach, D. et al. Loki: An Open-Source Tool for Fact Verification. — Open-source infrastructure for automated fact-checking pipelines.
  • Gigerenzer, G. & Brighton, H. (2009). Homo Heuristicus: Why Biased Minds Make Better Inferences. Topics in Cognitive Science, 1(1), 107–143. — The adaptive value of cognitive heuristics.
  • Mercier, H. & Sperber, D. (2011). Why Do Humans Reason? Arguments for an Argumentative Theory. Behavioral and Brain Sciences, 34(2), 57–74. — Reasoning as a social, argumentative faculty.
  • Wasserstein, R. L. & Lazar, N. A. (2016). The ASA Statement on p-Values. The American Statistician, 70(2), 129–133. — Official guidelines on the interpretation and misuse of p-values.

Encyclopedias & Digital Resources

Additional Research & Resources

  • Aristotle. Sophistical Refutations (De Sophisticis Elenchis). — The original classification of fallacies in Western philosophy (~350 BCE).
  • Cialdini, R. B. (2006). Influence: The Psychology of Persuasion. Rev. ed. Harper Business. — Six principles of persuasion and their exploitation.
  • Lakoff, G. (2004). Don't Think of an Elephant! Chelsea Green. — Framing and its role in political discourse.
  • Herman, E. S. & Chomsky, N. (1988). Manufacturing Consent: The Political Economy of the Mass Media. Pantheon. — The propaganda model of media analysis.
  • Ariely, D. (2008). Predictably Irrational. HarperCollins. — Systematic patterns in irrational decision-making.
  • Gigerenzer, G. (2002). Calculated Risks: How to Know When Numbers Deceive You. Simon & Schuster. — Statistical literacy and risk communication.
  • Stanovich, K. E. (2009). What Intelligence Tests Miss: The Psychology of Rational Thought. Yale University Press. — The distinction between intelligence and rationality.
  • Nisbett, R. E. & Ross, L. (1980). Human Inference: Strategies and Shortcomings of Social Judgment. Prentice-Hall. — Systematic errors in everyday reasoning.
  • Thaler, R. H. & Sunstein, C. R. (2008). Nudge: Improving Decisions About Health, Wealth, and Happiness. Yale University Press. — Bias-aware choice architecture.
  • Wardle, C. & Derakhshan, H. (2017). Information Disorder: Toward an interdisciplinary framework for research and policy making. Council of Europe Report DGI(2017)09. — Taxonomy of mis-, dis-, and mal-information.
  • Da San Martino, G. et al. (2019). Fine-Grained Analysis of Propaganda in News Articles. EMNLP 2019. — Span-level propaganda technique detection framework.
  • Boudry, M. et al. (2015). What Makes Weird Beliefs Thrive? The Epidemiology of Pseudoscience. Philosophical Psychology, 28(8). — How cognitive biases facilitate pseudoscientific reasoning.