Effects of examiner competency on the amblyopia screening performance of three visual acuity tests through Monte Carlo simulation

TY - GEN

T1 - Effects of examiner competency on the amblyopia screening performance of three visual acuity tests through Monte Carlo simulation

AU - Shand, Keely

AU - Boribalburephan, Atirut

AU - Giardini, Mario Ettore

N1 - Conference code: 47

PY - 2025/6/6

Y1 - 2025/6/6

N2 - Abstract— Amblyopia is amongst the leading causes of vision loss in children. This condition is reversible if diagnosed and treated early. Ideally, treatment should be instituted in early infancy, as the rate of success of treatment significantly drops after the age of seven. Screening for amblyopia in infants critically relies on examiner expertise, lending itself to errors. Automation and digitization of amblyopia screening tests is being actively pursued by the scientific community. Yet, for the known amblyopia screening protocols, little is known about the dependence of the relative performance on the error rate of an examiner. Yet, this information is key in designing an automated test. In the present paper we address the relationship between the examiner error rate (examiner competency) and the clinical performance of three major preferential-looking tests to screen for amblyopia, namely, Keeler Acuity Cards for Infants (KACI), Cardiff Acuity Targets (CAT), and Teller Acuity Cards (TAC). In our study, screening tests of the difference between visual acuity of both eyes were simulated, with the examiner error rate ranging from perfect testing to randomness. Amblyopia was defined as a difference of 0.2 logMAR between the eyes. The accuracy, precision, recall, False Negative Rate (FNR) and F1score were determined as a function of the examiner error rate, showing that all three protocols are dependent on the examiner competency, with CAT being the most dependent, due to the higher number of target presentations within a test. The best performing protocol was KACI, which exhibited high recall, important for screening purposes. Clinical Relevance— This paper assesses the relative effects of examiner-related clinical performance of three major preferential-looking protocols for screening for amblyopia, which is foundational to automating these tests.

AB - Abstract— Amblyopia is amongst the leading causes of vision loss in children. This condition is reversible if diagnosed and treated early. Ideally, treatment should be instituted in early infancy, as the rate of success of treatment significantly drops after the age of seven. Screening for amblyopia in infants critically relies on examiner expertise, lending itself to errors. Automation and digitization of amblyopia screening tests is being actively pursued by the scientific community. Yet, for the known amblyopia screening protocols, little is known about the dependence of the relative performance on the error rate of an examiner. Yet, this information is key in designing an automated test. In the present paper we address the relationship between the examiner error rate (examiner competency) and the clinical performance of three major preferential-looking tests to screen for amblyopia, namely, Keeler Acuity Cards for Infants (KACI), Cardiff Acuity Targets (CAT), and Teller Acuity Cards (TAC). In our study, screening tests of the difference between visual acuity of both eyes were simulated, with the examiner error rate ranging from perfect testing to randomness. Amblyopia was defined as a difference of 0.2 logMAR between the eyes. The accuracy, precision, recall, False Negative Rate (FNR) and F1score were determined as a function of the examiner error rate, showing that all three protocols are dependent on the examiner competency, with CAT being the most dependent, due to the higher number of target presentations within a test. The best performing protocol was KACI, which exhibited high recall, important for screening purposes. Clinical Relevance— This paper assesses the relative effects of examiner-related clinical performance of three major preferential-looking protocols for screening for amblyopia, which is foundational to automating these tests.

KW - Monte Carlo modeling

KW - ophthalmology

KW - amblyopia

KW - screening

M3 - Conference contribution

T3 - Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS

BT - 2025 47th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

PB - IEEE

T2 - 47th Annual International Conference of the IEEE Engineering in Medicine and Biology Society

Y2 - 14 June 2025 through 17 June 2025

ER -