Disagreement space in argument analysis

Annette Hautli-Janisz; Ella Schad; Chris Reed

Disagreement space in argument analysis

Annette Hautli-Janisz, Ella Schad, Chris Reed

Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

103 Downloads (Pure)

Abstract

For a highly subjective task such as recognising speaker intention and argumentation, the traditional way of generating gold standards is to aggregate a number of labels into a single one. However, this seriously neglects the underlying richness that characterises discourse and argumentation and is also, in some cases, straightforwardly impossible. In this paper, we present QT30nonaggr, the first corpus of non-aggregated argument annotation. QT30nonaggr encompasses 10% of QT30, the largest corpus of dialogical argumentation and analysed broadcast political debate currently available with 30 episodes of BBC’s ‘Question Time’ from 2020 and 2021. Based on a systematic and detailed investigation of annotation judgements across all steps of the annotation process, we structure the disagreement space with a taxonomy of the types of label disagreements in argument annotation, identifying the categories of annotation errors, fuzziness and ambiguity.

Original language	English
Title of host publication	Proceedings of the 1st Workshop on Perspectivist Approaches to NLP @LREC2022
Editors	Gavin Abercrombie, Valerio Basile, Sara Tonelli, Verena Rieser, Alexandra Uma
Place of Publication	Paris
Publisher	European Language Resources Association (ELRA)
Pages	1-9
Number of pages	9
ISBN (Electronic)	9791095546986
Publication status	Published - 2022
Event	1st Workshop on Perspectivist Approaches to NLP - Marseille, France Duration: 20 Jun 2022 → 20 Jun 2022

Conference

Conference	1st Workshop on Perspectivist Approaches to NLP
Abbreviated title	NLPerspectives 2022
Country/Territory	France
City	Marseille
Period	20/06/22 → 20/06/22

Keywords

argumentation and conflict
broadcast political debate
Inference Anchoring Theory
Question Time

ASJC Scopus subject areas

Information Systems and Management
Information Systems
Computer Science Applications

Access to Document

Final Published VersionFinal published version, 1.11 MBLicence: CC BY-NC

Cite this

@inproceedings{93528344e8c741cc809c4b1174b158ed,

title = "Disagreement space in argument analysis",

abstract = "For a highly subjective task such as recognising speaker intention and argumentation, the traditional way of generating gold standards is to aggregate a number of labels into a single one. However, this seriously neglects the underlying richness that characterises discourse and argumentation and is also, in some cases, straightforwardly impossible. In this paper, we present QT30nonaggr, the first corpus of non-aggregated argument annotation. QT30nonaggr encompasses 10% of QT30, the largest corpus of dialogical argumentation and analysed broadcast political debate currently available with 30 episodes of BBC{\textquoteright}s {\textquoteleft}Question Time{\textquoteright} from 2020 and 2021. Based on a systematic and detailed investigation of annotation judgements across all steps of the annotation process, we structure the disagreement space with a taxonomy of the types of label disagreements in argument annotation, identifying the categories of annotation errors, fuzziness and ambiguity.",

keywords = "argumentation and conflict, broadcast political debate, Inference Anchoring Theory, Question Time",

author = "Annette Hautli-Janisz and Ella Schad and Chris Reed",

note = "Publisher Copyright: {\textcopyright} European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0.; 1st Workshop on Perspectivist Approaches to NLP, NLPerspectives 2022 ; Conference date: 20-06-2022 Through 20-06-2022",

year = "2022",

language = "English",

pages = "1--9",

editor = "Gavin Abercrombie and Valerio Basile and Sara Tonelli and Verena Rieser and Alexandra Uma",

booktitle = "Proceedings of the 1st Workshop on Perspectivist Approaches to NLP @LREC2022",

publisher = "European Language Resources Association (ELRA)",

}

Disagreement space in argument analysis. / Hautli-Janisz, Annette; Schad, Ella ; Reed, Chris.
Proceedings of the 1st Workshop on Perspectivist Approaches to NLP @LREC2022. ed. / Gavin Abercrombie; Valerio Basile; Sara Tonelli; Verena Rieser; Alexandra Uma. Paris: European Language Resources Association (ELRA), 2022. p. 1-9.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Disagreement space in argument analysis

AU - Hautli-Janisz, Annette

AU - Schad, Ella

AU - Reed, Chris

N1 - Publisher Copyright: © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0.

PY - 2022

Y1 - 2022

N2 - For a highly subjective task such as recognising speaker intention and argumentation, the traditional way of generating gold standards is to aggregate a number of labels into a single one. However, this seriously neglects the underlying richness that characterises discourse and argumentation and is also, in some cases, straightforwardly impossible. In this paper, we present QT30nonaggr, the first corpus of non-aggregated argument annotation. QT30nonaggr encompasses 10% of QT30, the largest corpus of dialogical argumentation and analysed broadcast political debate currently available with 30 episodes of BBC’s ‘Question Time’ from 2020 and 2021. Based on a systematic and detailed investigation of annotation judgements across all steps of the annotation process, we structure the disagreement space with a taxonomy of the types of label disagreements in argument annotation, identifying the categories of annotation errors, fuzziness and ambiguity.

AB - For a highly subjective task such as recognising speaker intention and argumentation, the traditional way of generating gold standards is to aggregate a number of labels into a single one. However, this seriously neglects the underlying richness that characterises discourse and argumentation and is also, in some cases, straightforwardly impossible. In this paper, we present QT30nonaggr, the first corpus of non-aggregated argument annotation. QT30nonaggr encompasses 10% of QT30, the largest corpus of dialogical argumentation and analysed broadcast political debate currently available with 30 episodes of BBC’s ‘Question Time’ from 2020 and 2021. Based on a systematic and detailed investigation of annotation judgements across all steps of the annotation process, we structure the disagreement space with a taxonomy of the types of label disagreements in argument annotation, identifying the categories of annotation errors, fuzziness and ambiguity.

KW - argumentation and conflict

KW - broadcast political debate

KW - Inference Anchoring Theory

KW - Question Time

UR - http://www.scopus.com/inward/record.url?scp=85145883100&partnerID=8YFLogxK

UR - http://www.lrec-conf.org/proceedings/lrec2022/workshops/NLPerspectives/index.html

M3 - Conference contribution

AN - SCOPUS:85145883100

SP - 1

EP - 9

BT - Proceedings of the 1st Workshop on Perspectivist Approaches to NLP @LREC2022

A2 - Abercrombie, Gavin

A2 - Basile, Valerio

A2 - Tonelli, Sara

A2 - Rieser, Verena

A2 - Uma, Alexandra

PB - European Language Resources Association (ELRA)

CY - Paris

T2 - 1st Workshop on Perspectivist Approaches to NLP

Y2 - 20 June 2022 through 20 June 2022

ER -

Disagreement space in argument analysis

Abstract

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this