Validating retinal fundus image analysis algorithms: issues and a proposal

Emanuele Trucco; Alfredo Ruggeri; Thomas Karnowski; Luca Giancardo; Edward Chaum; Jean Pierre Hubschman; Bashir al-Diri; Carol Y. Cheung; Damon Wong; Michael Abramoff; Gilbert Lim; Dinesh Kumar; Philippe Burlina; Neil M. Bressler; Herbert F. Jelinek; Fabrice Meriaudeau; Gwenole Quellec; Tom MacGillivray; Bal Dhillon

doi:10.1167/iovs.12-10347

Validating retinal fundus image analysis algorithms: issues and a proposal

Emanuele Trucco, Alfredo Ruggeri, Thomas Karnowski, Luca Giancardo, Edward Chaum, Jean Pierre Hubschman, Bashir al-Diri, Carol Y. Cheung, Damon Wong, Michael Abramoff, Gilbert Lim, Dinesh Kumar, Philippe Burlina, Neil M. Bressler, Herbert F. Jelinek, Fabrice Meriaudeau, Gwenole Quellec, Tom MacGillivray, Bal Dhillon

Research output: Contribution to journal › Article › peer-review

144 Citations (Scopus)

Abstract

This paper concerns the validation of automatic retinal image analysis (ARIA) algorithms. For reasons of space and consistency, we concentrate on the validation of algorithms processing color fundus camera images, currently the largest section of the ARIA literature. We sketch the context (imaging instruments and target tasks) of ARIA validation, summarizing the main image analysis and validation techniques. We then present a list of recommendations focusing on the creation of large repositories of test data created by international consortia, easily accessible via moderated Web sites, including multicenter annotations by multiple experts, specific to clinical tasks, and capable of running submitted software automatically on the data stored, with clear and widely agreed-on performance criteria, to provide a fair comparison.

Original language	English
Pages (from-to)	3546-3559
Number of pages	14
Journal	Investigative Ophthalmology & Visual Science
Volume	54
Issue number	5
DOIs	https://doi.org/10.1167/iovs.12-10347
Publication status	Published - 24 May 2013

Keywords

AUTOMATED DETECTION
VESSEL SEGMENTATION
validation
GLAUCOMA ANALYSIS
fundus image analysis
TO-DISC RATIO
COST-EFFECTIVENESS
MACULAR DEGENERATION
CARDIAC CYCLE
OPTIC DISC
DIABETIC-RETINOPATHY
reference standards
SCREENING TOOL

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1167/iovs.12-10347

Cite this

Trucco, E., Ruggeri, A., Karnowski, T., Giancardo, L., Chaum, E., Hubschman, J. P., al-Diri, B., Cheung, C. Y., Wong, D., Abramoff, M., Lim, G., Kumar, D., Burlina, P., Bressler, N. M., Jelinek, H. F., Meriaudeau, F., Quellec, G., MacGillivray, T., & Dhillon, B. (2013). Validating retinal fundus image analysis algorithms: issues and a proposal. Investigative Ophthalmology & Visual Science, 54(5), 3546-3559. https://doi.org/10.1167/iovs.12-10347

@article{33e6776ec0ca439ba47e9b3d13f0a99a,

title = "Validating retinal fundus image analysis algorithms: issues and a proposal",

abstract = "This paper concerns the validation of automatic retinal image analysis (ARIA) algorithms. For reasons of space and consistency, we concentrate on the validation of algorithms processing color fundus camera images, currently the largest section of the ARIA literature. We sketch the context (imaging instruments and target tasks) of ARIA validation, summarizing the main image analysis and validation techniques. We then present a list of recommendations focusing on the creation of large repositories of test data created by international consortia, easily accessible via moderated Web sites, including multicenter annotations by multiple experts, specific to clinical tasks, and capable of running submitted software automatically on the data stored, with clear and widely agreed-on performance criteria, to provide a fair comparison.",

keywords = "AUTOMATED DETECTION, VESSEL SEGMENTATION, validation, GLAUCOMA ANALYSIS, fundus image analysis, TO-DISC RATIO, COST-EFFECTIVENESS, MACULAR DEGENERATION, CARDIAC CYCLE, OPTIC DISC, DIABETIC-RETINOPATHY, reference standards, SCREENING TOOL",

author = "Emanuele Trucco and Alfredo Ruggeri and Thomas Karnowski and Luca Giancardo and Edward Chaum and Hubschman, {Jean Pierre} and Bashir al-Diri and Cheung, {Carol Y.} and Damon Wong and Michael Abramoff and Gilbert Lim and Dinesh Kumar and Philippe Burlina and Bressler, {Neil M.} and Jelinek, {Herbert F.} and Fabrice Meriaudeau and Gwenole Quellec and Tom MacGillivray and Bal Dhillon",

year = "2013",

month = may,

day = "24",

doi = "10.1167/iovs.12-10347",

language = "English",

volume = "54",

pages = "3546--3559",

journal = "Investigative Ophthalmology & Visual Science",

issn = "0146-0404",

publisher = "Association for Research in Vision and Ophthalmology Inc.",

number = "5",

}

Trucco, E, Ruggeri, A, Karnowski, T, Giancardo, L, Chaum, E, Hubschman, JP, al-Diri, B, Cheung, CY, Wong, D, Abramoff, M, Lim, G, Kumar, D, Burlina, P, Bressler, NM, Jelinek, HF, Meriaudeau, F, Quellec, G, MacGillivray, T & Dhillon, B 2013, 'Validating retinal fundus image analysis algorithms: issues and a proposal', Investigative Ophthalmology & Visual Science, vol. 54, no. 5, pp. 3546-3559. https://doi.org/10.1167/iovs.12-10347

TY - JOUR

T1 - Validating retinal fundus image analysis algorithms

T2 - issues and a proposal

AU - Trucco, Emanuele

AU - Ruggeri, Alfredo

AU - Karnowski, Thomas

AU - Giancardo, Luca

AU - Chaum, Edward

AU - Hubschman, Jean Pierre

AU - al-Diri, Bashir

AU - Cheung, Carol Y.

AU - Wong, Damon

AU - Abramoff, Michael

AU - Lim, Gilbert

AU - Kumar, Dinesh

AU - Burlina, Philippe

AU - Bressler, Neil M.

AU - Jelinek, Herbert F.

AU - Meriaudeau, Fabrice

AU - Quellec, Gwenole

AU - MacGillivray, Tom

AU - Dhillon, Bal

PY - 2013/5/24

Y1 - 2013/5/24

N2 - This paper concerns the validation of automatic retinal image analysis (ARIA) algorithms. For reasons of space and consistency, we concentrate on the validation of algorithms processing color fundus camera images, currently the largest section of the ARIA literature. We sketch the context (imaging instruments and target tasks) of ARIA validation, summarizing the main image analysis and validation techniques. We then present a list of recommendations focusing on the creation of large repositories of test data created by international consortia, easily accessible via moderated Web sites, including multicenter annotations by multiple experts, specific to clinical tasks, and capable of running submitted software automatically on the data stored, with clear and widely agreed-on performance criteria, to provide a fair comparison.

AB - This paper concerns the validation of automatic retinal image analysis (ARIA) algorithms. For reasons of space and consistency, we concentrate on the validation of algorithms processing color fundus camera images, currently the largest section of the ARIA literature. We sketch the context (imaging instruments and target tasks) of ARIA validation, summarizing the main image analysis and validation techniques. We then present a list of recommendations focusing on the creation of large repositories of test data created by international consortia, easily accessible via moderated Web sites, including multicenter annotations by multiple experts, specific to clinical tasks, and capable of running submitted software automatically on the data stored, with clear and widely agreed-on performance criteria, to provide a fair comparison.

KW - AUTOMATED DETECTION

KW - VESSEL SEGMENTATION

KW - validation

KW - GLAUCOMA ANALYSIS

KW - fundus image analysis

KW - TO-DISC RATIO

KW - COST-EFFECTIVENESS

KW - MACULAR DEGENERATION

KW - CARDIAC CYCLE

KW - OPTIC DISC

KW - DIABETIC-RETINOPATHY

KW - reference standards

KW - SCREENING TOOL

UR - http://www.scopus.com/inward/record.url?scp=84883286209&partnerID=8YFLogxK

U2 - 10.1167/iovs.12-10347

DO - 10.1167/iovs.12-10347

M3 - Article

SN - 0146-0404

VL - 54

SP - 3546

EP - 3559

JO - Investigative Ophthalmology & Visual Science

JF - Investigative Ophthalmology & Visual Science

IS - 5

ER -

Validating retinal fundus image analysis algorithms: issues and a proposal

Abstract

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this