The CASS Technique for Evaluating the Performance of Argument Mining

Research output: Chapter in Book/Report/Conference proceedingChapter (peer-reviewed)

Abstract

Argument mining integrates many distinct computational linguistics tasks, and as a result, reporting agreement between annotators or between automated output and gold standard is particularly challenging. More worrying for the field, agreement and performance are also reported in a wide variety of different ways, making comparison between approaches difficult. To solve this problem, we propose the CASS technique for combining metrics covering different parts of the argument mining task. CASS delivers a justified
method of integrating results yielding confusion matrices from which CASS-κ and CASS-F1 scores can be calculated.
Original languageEnglish
Title of host publicationProceedings of the 3rd Workshop on Argument Mining (ArgMining2016)
PublisherAssociation for Computational Linguistics
Pages40-49
Number of pages10
ISBN (Electronic)9781945626173
DOIs
Publication statusPublished - Aug 2016
Event3rd Workshop on Argument Mining - Berlin, Germany
Duration: 12 Aug 201612 Aug 2016
http://argmining2016.arg.tech/

Conference

Conference3rd Workshop on Argument Mining
CountryGermany
CityBerlin
Period12/08/1612/08/16
Internet address

Fingerprint Dive into the research topics of 'The CASS Technique for Evaluating the Performance of Argument Mining'. Together they form a unique fingerprint.

  • Cite this

    Duthie, R., Lawrence, J., Budzynska, K., & Reed, C. (2016). The CASS Technique for Evaluating the Performance of Argument Mining. In Proceedings of the 3rd Workshop on Argument Mining (ArgMining2016) (pp. 40-49). [W16-2805] Association for Computational Linguistics. https://doi.org/10.18653/v1/W16-2805