Annotating and Mining Hypotheses in Argumentation

TY - GEN

T1 - Annotating and Mining Hypotheses in Argumentation

AU - Schad, Ella

AU - Gorska, Kamila

AU - Maguire, Eimear

AU - Ruiz-Dolz, Ramon

AU - Abraham, Melvin

AU - Lawrence, John

AU - Visser, Jacky

N1 - Conference code: 10

PY - 2024

Y1 - 2024

N2 - Generating and evaluating hypotheses about past, present, and future events is core to argumentation in many domains, such as forensic investigations, medical diagnostics, and scientific research. In this paper, we explore the role of hypothesis-making in argumentative dialogue. To do so, we introduce an annotated dataset of 502 hypotheses in the existing RIP corpus of collaborative problem-solving in murder mystery games, creating the RIP1 corpus. Propositions marked as hypotheses in RIP1 correlate systematically with argument structure (previously annotated according to Inference Anchoring Theory). We explore the interaction between arguments and hypotheses, showing hypotheses are often conclusions of arguments and differences between how hypotheses and assertions are treated in dialogue. Based on this quantitative analysis, we conduct preliminary computational experiments establishing a baseline for the automatic mining of hypotheses. Experimentation with a Support Vector Machine and a fine-tuned RoBERTa model shows initial performance on text span classification with an F1 score of 80.0, outperforming random and majority baselines, and providing a target for future improvement.

AB - Generating and evaluating hypotheses about past, present, and future events is core to argumentation in many domains, such as forensic investigations, medical diagnostics, and scientific research. In this paper, we explore the role of hypothesis-making in argumentative dialogue. To do so, we introduce an annotated dataset of 502 hypotheses in the existing RIP corpus of collaborative problem-solving in murder mystery games, creating the RIP1 corpus. Propositions marked as hypotheses in RIP1 correlate systematically with argument structure (previously annotated according to Inference Anchoring Theory). We explore the interaction between arguments and hypotheses, showing hypotheses are often conclusions of arguments and differences between how hypotheses and assertions are treated in dialogue. Based on this quantitative analysis, we conduct preliminary computational experiments establishing a baseline for the automatic mining of hypotheses. Experimentation with a Support Vector Machine and a fine-tuned RoBERTa model shows initial performance on text span classification with an F1 score of 80.0, outperforming random and majority baselines, and providing a target for future improvement.

KW - Collaborative problem-solving

KW - Dialogue

KW - Hypothesis

KW - Text Mining

UR - https://ebooks.iospress.nl/volume/computational-models-of-argument-proceedings-of-comma-2024

U2 - 10.3233/FAIA240326

DO - 10.3233/FAIA240326

M3 - Conference contribution

SN - 9781643685342

T3 - Frontiers in Artificial Intelligence and Applications

SP - 253

EP - 264

BT - Computational Models of Argument

A2 - Reed, Chris

A2 - Thimm, Matthias

A2 - Rienstra, Tjitze

PB - IOS Press

T2 - The 10th International Conference on Computational Models of Argument

Y2 - 18 September 2024 through 20 September 2024

ER -