Calibration of Deep Medical Image Classifiers: An Empirical Comparison using Dermatology and Histopathology Datasets

Jacob Carse, Andres Alvarez Olmo, Stephen McKenna

Research output: Chapter in Book/Report/Conference proceedingConference contribution

44 Downloads (Pure)

Abstract

As deep learning classifiers become ever more widely deployed for medical image analysis tasks, issues of predictive calibration need to be addressed. Mis-calibration is the deviation between predictive probability (confidence) and classification correctness. Well-calibrated classifiers enable cost-sensitive and selective decision-making. This paper presents an empirical investigation of calibration methods on two medical image datasets (multi-class dermatology and binary histopathology image classification). We show the effect of temperature scaling with temperature optimized using various measures of calibration replacing the standard negative log-likelihood. We do so not only for networks trained using one-hot encoding and cross-entropy loss, but also using focal loss and label smoothing. We compare these with two Bayesian methods. Results suggest little or no advantage to the use of alternative calibration metrics for tuning temperature. Temperature scaling of networks trained using focal loss (with appropriate hyperparameters) provided strong results in terms of both calibration and accuracy across both datasets
Original languageEnglish
Title of host publicationMICCAI Satellite Events joint LNCS proceedings
PublisherSpringer Nature
Publication statusAccepted/In press - 24 Jul 2022
EventUncertainty for Safe Utilization of Machine Learning in Medical Imaging - , Singapore
Duration: 22 Sep 202222 Sep 2022
https://unsuremiccai.github.io/

Conference

ConferenceUncertainty for Safe Utilization of Machine Learning in Medical Imaging
Abbreviated titleUNSURE2022
Country/TerritorySingapore
Period22/09/2222/09/22
Internet address

Keywords

  • Calibration
  • Classification
  • Uncertainty
  • Deep Learning

Fingerprint

Dive into the research topics of 'Calibration of Deep Medical Image Classifiers: An Empirical Comparison using Dermatology and Histopathology Datasets'. Together they form a unique fingerprint.

Cite this