Robust Selective Classification of Skin Lesions with Asymmetric Costs

Jacob Carse; Tamas Suveges; Stephen Hogg; Emanuele Trucco; Charlotte Proby; Colin Fleming; Stephen McKenna

doi:10.1007/978-3-030-87735-4_11

Robust Selective Classification of Skin Lesions with Asymmetric Costs

Jacob Carse, Tamas Suveges, Stephen Hogg, Emanuele Trucco, Charlotte Proby, Colin Fleming, Stephen McKenna (Lead / Corresponding author)

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Citation (Scopus)

239 Downloads (Pure)

Abstract

Automated image analysis of skin lesions has potential to improve diagnostic decision making. A clinically useful system should be selective, rejecting images it is ill-equipped to classify, for example because they are of lesion types not represented well in training data. Furthermore, lesion classifiers should support cost-sensitive decision making. We investigate methods for selective, cost-sensitive classification of lesions as benign or malignant using test images of lesion types represented and not represented in training data. We propose EC-SelectiveNet, a modification to SelectiveNet that discards the selection head at test time, making decisions based on expected costs instead. Experiments show that training for full coverage is beneficial even when operating at lower coverage, and that EC-SelectiveNet outperforms standard cross-entropy training, whether or not temperature scaling or Monte Carlo dropout averaging are used, in both symmetric and asymmetric cost settings.

Original language	English
Title of host publication	Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis - 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021, Held in Conjunction with MICCAI 2021, Proceedings
Editors	Carole H. Sudre, Roxane Licandro, Christian Baumgartner, Andrew Melbourne, Adrian Dalca, Jana Hutter, Ryutaro Tanno, Esra Abaci Turk, Koen Van Leemput, Jordina Torrents Barrena, William M. Wells, Christopher Macgowan
Place of Publication	Cham
Publisher	Springer
Pages	112-121
Number of pages	10
ISBN (Electronic)	9783030877354
ISBN (Print)	9783030877347
DOIs	https://doi.org/10.1007/978-3-030-87735-4_11
Publication status	Published - 2021

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12959 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1007/978-3-030-87735-4_11Licence: No Licence / Unknown

Author Accepted ManuscriptAccepted author manuscript, 616 KB

Cite this

Carse, J., Suveges, T., Hogg, S., Trucco, E., Proby, C., Fleming, C., & McKenna, S. (2021). Robust Selective Classification of Skin Lesions with Asymmetric Costs. In C. H. Sudre, R. Licandro, C. Baumgartner, A. Melbourne, A. Dalca, J. Hutter, R. Tanno, E. Abaci Turk, K. Van Leemput, J. Torrents Barrena, W. M. Wells, & C. Macgowan (Eds.), Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis - 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021, Held in Conjunction with MICCAI 2021, Proceedings (pp. 112-121). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12959 LNCS). Springer . https://doi.org/10.1007/978-3-030-87735-4_11

Carse, Jacob ; Suveges, Tamas ; Hogg, Stephen et al. / Robust Selective Classification of Skin Lesions with Asymmetric Costs. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis - 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021, Held in Conjunction with MICCAI 2021, Proceedings. editor / Carole H. Sudre ; Roxane Licandro ; Christian Baumgartner ; Andrew Melbourne ; Adrian Dalca ; Jana Hutter ; Ryutaro Tanno ; Esra Abaci Turk ; Koen Van Leemput ; Jordina Torrents Barrena ; William M. Wells ; Christopher Macgowan. Cham : Springer , 2021. pp. 112-121 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{67a58f6173f143ad95a32c0b83d10d39,

title = "Robust Selective Classification of Skin Lesions with Asymmetric Costs",

abstract = "Automated image analysis of skin lesions has potential to improve diagnostic decision making. A clinically useful system should be selective, rejecting images it is ill-equipped to classify, for example because they are of lesion types not represented well in training data. Furthermore, lesion classifiers should support cost-sensitive decision making. We investigate methods for selective, cost-sensitive classification of lesions as benign or malignant using test images of lesion types represented and not represented in training data. We propose EC-SelectiveNet, a modification to SelectiveNet that discards the selection head at test time, making decisions based on expected costs instead. Experiments show that training for full coverage is beneficial even when operating at lower coverage, and that EC-SelectiveNet outperforms standard cross-entropy training, whether or not temperature scaling or Monte Carlo dropout averaging are used, in both symmetric and asymmetric cost settings.",

author = "Jacob Carse and Tamas Suveges and Stephen Hogg and Emanuele Trucco and Charlotte Proby and Colin Fleming and Stephen McKenna",

note = "This paper reports independent research funded by the National Institute for Health Research (Artificial Intelligence, Deep learning for effective triaging of skin disease in the NHS, AI AWARD01901) and NHSX. The views expressed in this publication are those of the authors and not necessarily those of the National Institute for Health Research, NHSX or the Department of Health and Social Care. This research was also funded by the Detect Cancer Early programme, and the Discovery Institute of Dermatology.",

year = "2021",

doi = "10.1007/978-3-030-87735-4_11",

language = "English",

isbn = "9783030877347",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer ",

pages = "112--121",

editor = "Sudre, {Carole H.} and Roxane Licandro and Christian Baumgartner and Andrew Melbourne and Adrian Dalca and Jana Hutter and Ryutaro Tanno and {Abaci Turk}, Esra and {Van Leemput}, Koen and {Torrents Barrena}, Jordina and Wells, {William M.} and Christopher Macgowan",

booktitle = "Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis - 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021, Held in Conjunction with MICCAI 2021, Proceedings",

}

Carse, J, Suveges, T, Hogg, S, Trucco, E , Proby, C, Fleming, C & McKenna, S 2021, Robust Selective Classification of Skin Lesions with Asymmetric Costs. in CH Sudre, R Licandro, C Baumgartner, A Melbourne, A Dalca, J Hutter, R Tanno, E Abaci Turk, K Van Leemput, J Torrents Barrena, WM Wells & C Macgowan (eds), Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis - 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021, Held in Conjunction with MICCAI 2021, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12959 LNCS, Springer , Cham, pp. 112-121. https://doi.org/10.1007/978-3-030-87735-4_11

Robust Selective Classification of Skin Lesions with Asymmetric Costs. / Carse, Jacob; Suveges, Tamas; Hogg, Stephen et al.
Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis - 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021, Held in Conjunction with MICCAI 2021, Proceedings. ed. / Carole H. Sudre; Roxane Licandro; Christian Baumgartner; Andrew Melbourne; Adrian Dalca; Jana Hutter; Ryutaro Tanno; Esra Abaci Turk; Koen Van Leemput; Jordina Torrents Barrena; William M. Wells; Christopher Macgowan. Cham: Springer , 2021. p. 112-121 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12959 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Robust Selective Classification of Skin Lesions with Asymmetric Costs

AU - Carse, Jacob

AU - Suveges, Tamas

AU - Hogg, Stephen

AU - Trucco, Emanuele

AU - Proby, Charlotte

AU - Fleming, Colin

AU - McKenna, Stephen

N1 - This paper reports independent research funded by the National Institute for Health Research (Artificial Intelligence, Deep learning for effective triaging of skin disease in the NHS, AI AWARD01901) and NHSX. The views expressed in this publication are those of the authors and not necessarily those of the National Institute for Health Research, NHSX or the Department of Health and Social Care. This research was also funded by the Detect Cancer Early programme, and the Discovery Institute of Dermatology.

PY - 2021

Y1 - 2021

N2 - Automated image analysis of skin lesions has potential to improve diagnostic decision making. A clinically useful system should be selective, rejecting images it is ill-equipped to classify, for example because they are of lesion types not represented well in training data. Furthermore, lesion classifiers should support cost-sensitive decision making. We investigate methods for selective, cost-sensitive classification of lesions as benign or malignant using test images of lesion types represented and not represented in training data. We propose EC-SelectiveNet, a modification to SelectiveNet that discards the selection head at test time, making decisions based on expected costs instead. Experiments show that training for full coverage is beneficial even when operating at lower coverage, and that EC-SelectiveNet outperforms standard cross-entropy training, whether or not temperature scaling or Monte Carlo dropout averaging are used, in both symmetric and asymmetric cost settings.

AB - Automated image analysis of skin lesions has potential to improve diagnostic decision making. A clinically useful system should be selective, rejecting images it is ill-equipped to classify, for example because they are of lesion types not represented well in training data. Furthermore, lesion classifiers should support cost-sensitive decision making. We investigate methods for selective, cost-sensitive classification of lesions as benign or malignant using test images of lesion types represented and not represented in training data. We propose EC-SelectiveNet, a modification to SelectiveNet that discards the selection head at test time, making decisions based on expected costs instead. Experiments show that training for full coverage is beneficial even when operating at lower coverage, and that EC-SelectiveNet outperforms standard cross-entropy training, whether or not temperature scaling or Monte Carlo dropout averaging are used, in both symmetric and asymmetric cost settings.

UR - http://www.scopus.com/inward/record.url?scp=85117118967&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-87735-4_11

DO - 10.1007/978-3-030-87735-4_11

M3 - Conference contribution

SN - 9783030877347

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 112

EP - 121

BT - Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis - 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021, Held in Conjunction with MICCAI 2021, Proceedings

A2 - Sudre, Carole H.

A2 - Licandro, Roxane

A2 - Baumgartner, Christian

A2 - Melbourne, Andrew

A2 - Dalca, Adrian

A2 - Hutter, Jana

A2 - Tanno, Ryutaro

A2 - Abaci Turk, Esra

A2 - Van Leemput, Koen

A2 - Torrents Barrena, Jordina

A2 - Wells, William M.

A2 - Macgowan, Christopher

PB - Springer

CY - Cham

ER -

Carse J, Suveges T, Hogg S, Trucco E , Proby C, Fleming C et al. Robust Selective Classification of Skin Lesions with Asymmetric Costs. In Sudre CH, Licandro R, Baumgartner C, Melbourne A, Dalca A, Hutter J, Tanno R, Abaci Turk E, Van Leemput K, Torrents Barrena J, Wells WM, Macgowan C, editors, Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis - 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021, Held in Conjunction with MICCAI 2021, Proceedings. Cham: Springer . 2021. p. 112-121. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Epub 2021 Sept 25. doi: 10.1007/978-3-030-87735-4_11

Robust Selective Classification of Skin Lesions with Asymmetric Costs

Abstract

Publication series

ASJC Scopus subject areas

UN SDGs

Access to Document

Other files and links

Fingerprint

Deep Learning for effective triaging of skin disease in the NHS

Cite this

Robust Selective Classification of Skin Lesions with Asymmetric Costs

Abstract

Publication series

ASJC Scopus subject areas

UN SDGs

Access to Document

Other files and links

Fingerprint

Projects

Deep Learning for effective triaging of skin disease in the NHS

Cite this