Abstract
As deep learning classifiers become ever more widely deployed for medical image analysis tasks, issues of predictive calibration need to be addressed. Mis-calibration is the deviation between predictive probability (confidence) and classification correctness. Well-calibrated classifiers enable cost-sensitive and selective decision-making. This paper presents an empirical investigation of calibration methods on two medical image datasets (multi-class dermatology and binary histopathology image classification). We show the effect of temperature scaling with temperature optimized using various measures of calibration replacing the standard negative log-likelihood. We do so not only for networks trained using one-hot encoding and cross-entropy loss, but also using focal loss and label smoothing. We compare these with two Bayesian methods. Results suggest little or no advantage to the use of alternative calibration metrics for tuning temperature. Temperature scaling of networks trained using focal loss (with appropriate hyperparameters) provided strong results in terms of both calibration and accuracy across both datasets.
Original language | English |
---|---|
Title of host publication | Uncertainty for Safe Utilization of Machine Learning in Medical Imaging |
Subtitle of host publication | 4th International Workshop, UNSURE 2022, Held in Conjunction with MICCAI 2022, Singapore, September 18, 2022, Proceedings |
Editors | Carole H. Sudre, Christian F. Baumgartner, Adrian Dalca, Chen Qin, Ryutaro Tanno, Koen Van Leemput, William M. Wells III |
Place of Publication | Switzerland |
Publisher | Springer Nature |
Pages | 89-99 |
Number of pages | 11 |
Edition | 1 |
ISBN (Electronic) | 9783031167492 |
ISBN (Print) | 9783031167485 |
DOIs | |
Publication status | Published - 2022 |
Event | Uncertainty for Safe Utilization of Machine Learning in Medical Imaging - , Singapore Duration: 22 Sept 2022 → 22 Sept 2022 https://unsuremiccai.github.io/ |
Publication series
Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
---|---|
Volume | 13563 LNCS |
ISSN (Print) | 0302-9743 |
ISSN (Electronic) | 1611-3349 |
Conference
Conference | Uncertainty for Safe Utilization of Machine Learning in Medical Imaging |
---|---|
Abbreviated title | UNSURE2022 |
Country/Territory | Singapore |
Period | 22/09/22 → 22/09/22 |
Internet address |
Keywords
- Calibration
- Classification
- Deep learning
- Uncertainty
ASJC Scopus subject areas
- Theoretical Computer Science
- General Computer Science