Calibrating Where It Matters: Constrained Temperature Scaling

Stephen McKenna; Jacob Carse

Calibrating Where It Matters: Constrained Temperature Scaling

Computing

Research output: Contribution to conference › Paper › peer-review

Abstract

We consider calibration of convolutional classifiers for diagnostic decision making. Clinical decision makers can use calibrated classifiers to minimise expected costs given their own cost function. Such functions are usually unknown at training time. If minimising expected costs is the primary aim, algorithms should focus on tuning calibration in regions of probability simplex likely to effect decisions. We give an example, modifying temperature scaling calibration, and demonstrate improved calibration where it matters using convnets trained to classify dermoscopy images.

Original language	English
Number of pages	4
Publication status	Published - 16 Dec 2023
Event	Medical Imaging Meets NeurIPS: An official NeurIPS Workshop - Ernest N. Morial Convention Center, New Orleans, United States Duration: 16 Dec 2023 → 16 Dec 2023 https://sites.google.com/view/med-neurips2023/submissions?authuser=0

Workshop

Workshop	Medical Imaging Meets NeurIPS
Country/Territory	United States
City	New Orleans
Period	16/12/23 → 16/12/23
Internet address	https://sites.google.com/view/med-neurips2023/submissions?authuser=0

Cite this

@conference{1cf3fcf5ac064ca39fda5490318e4f45,

title = "Calibrating Where It Matters: Constrained Temperature Scaling",

abstract = "We consider calibration of convolutional classifiers for diagnostic decision making. Clinical decision makers can use calibrated classifiers to minimise expected costs given their own cost function. Such functions are usually unknown at training time. If minimising expected costs is the primary aim, algorithms should focus on tuning calibration in regions of probability simplex likely to effect decisions. We give an example, modifying temperature scaling calibration, and demonstrate improved calibration where it matters using convnets trained to classify dermoscopy images.",

author = "Stephen McKenna and Jacob Carse",

year = "2023",

month = dec,

day = "16",

language = "English",

note = "Medical Imaging Meets NeurIPS : An official NeurIPS Workshop ; Conference date: 16-12-2023 Through 16-12-2023",

url = "https://sites.google.com/view/med-neurips2023/submissions?authuser=0",

}

TY - CONF

T1 - Calibrating Where It Matters

T2 - Medical Imaging Meets NeurIPS

AU - McKenna, Stephen

AU - Carse, Jacob

PY - 2023/12/16

Y1 - 2023/12/16

N2 - We consider calibration of convolutional classifiers for diagnostic decision making. Clinical decision makers can use calibrated classifiers to minimise expected costs given their own cost function. Such functions are usually unknown at training time. If minimising expected costs is the primary aim, algorithms should focus on tuning calibration in regions of probability simplex likely to effect decisions. We give an example, modifying temperature scaling calibration, and demonstrate improved calibration where it matters using convnets trained to classify dermoscopy images.

AB - We consider calibration of convolutional classifiers for diagnostic decision making. Clinical decision makers can use calibrated classifiers to minimise expected costs given their own cost function. Such functions are usually unknown at training time. If minimising expected costs is the primary aim, algorithms should focus on tuning calibration in regions of probability simplex likely to effect decisions. We give an example, modifying temperature scaling calibration, and demonstrate improved calibration where it matters using convnets trained to classify dermoscopy images.

UR - https://sites.google.com/view/med-neurips2023/home

M3 - Paper

Y2 - 16 December 2023 through 16 December 2023

ER -

Calibrating Where It Matters: Constrained Temperature Scaling

Abstract

Workshop

Other files and links

Fingerprint

Cite this