TY - JOUR
T1 - Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans
AU - Roberts, Michael
AU - Driggs, Derek
AU - Thorpe, Matthew
AU - Gilbey, Julian
AU - Yeung, Michael
AU - Ursprung, Stephan
AU - Aviles-Rivero, Angelica I.
AU - Etmann, Christian
AU - McCague, Cathal
AU - Beer, Lucian
AU - Weir-McCall, Jonathan R.
AU - Teng, Zhongzhao
AU - Gkrania-Klotsas, Effrossyni
AU - AIX-COVNET
AU - Ruggiero, Alessandro
AU - Korhonen, Anna
AU - Jefferson, Emily
AU - Ako, Emmanuel
AU - Langs, Georg
AU - Gozaliasl, Ghassem
AU - Yang, Guang
AU - Prosch, Helmut
AU - Preller, Jacobus
AU - Stanczuk, Jan
AU - Tang, Jing
AU - Hofmanninger, Johannes
AU - Babar, Judith
AU - Sánchez, Lorena Escudero
AU - Thillai, Muhunthan
AU - Gonzalez, Paula Martin
AU - Teare, Philip
AU - Zhu, Xiaoxiang
AU - Patel, Mishal
AU - Cafolla, Conor
AU - Azadbakht, Hojjat
AU - Jacob, Joseph
AU - Lowe, Josh
AU - Zhang, Kang
AU - Bradley, Kyle
AU - Wassin, Marcel
AU - Holzer, Markus
AU - Ji, Kangyu
AU - Ortet, Maria Delgado
AU - Ai, Tao
AU - Walton, Nicholas
AU - Lio, Pietro
AU - Stranks, Samuel
AU - Shadbahr, Tolou
AU - Lin, Weizhe
AU - Zha, Yunfei
AU - Niu, Zhangming
AU - Rudd, James H. F.
AU - Sala, Evis
AU - Schönlieb, Carola-Bibiane
N1 - Publisher Copyright:
© 2021, The Author(s).
Copyright:
Copyright 2021 Elsevier B.V., All rights reserved.
PY - 2021/3
Y1 - 2021/3
N2 - Machine learning methods offer great promise for fast and accurate detection and prognostication of coronavirus disease 2019 (COVID-19) from standard-of-care chest radiographs (CXR) and chest computed tomography (CT) images. Many articles have been published in 2020 describing new machine learning-based models for both of these tasks, but it is unclear which are of potential clinical utility. In this systematic review, we consider all published papers and preprints, for the period from 1 January 2020 to 3 October 2020, which describe new machine learning models for the diagnosis or prognosis of COVID-19 from CXR or CT images. All manuscripts uploaded to bioRxiv, medRxiv and arXiv along with all entries in EMBASE and MEDLINE in this timeframe are considered. Our search identified 2,212 studies, of which 415 were included after initial screening and, after quality screening, 62 studies were included in this systematic review. Our review finds that none of the models identified are of potential clinical use due to methodological flaws and/or underlying biases. This is a major weakness, given the urgency with which validated COVID-19 models are needed. To address this, we give many recommendations which, if followed, will solve these issues and lead to higher-quality model development and well-documented manuscripts.
AB - Machine learning methods offer great promise for fast and accurate detection and prognostication of coronavirus disease 2019 (COVID-19) from standard-of-care chest radiographs (CXR) and chest computed tomography (CT) images. Many articles have been published in 2020 describing new machine learning-based models for both of these tasks, but it is unclear which are of potential clinical utility. In this systematic review, we consider all published papers and preprints, for the period from 1 January 2020 to 3 October 2020, which describe new machine learning models for the diagnosis or prognosis of COVID-19 from CXR or CT images. All manuscripts uploaded to bioRxiv, medRxiv and arXiv along with all entries in EMBASE and MEDLINE in this timeframe are considered. Our search identified 2,212 studies, of which 415 were included after initial screening and, after quality screening, 62 studies were included in this systematic review. Our review finds that none of the models identified are of potential clinical use due to methodological flaws and/or underlying biases. This is a major weakness, given the urgency with which validated COVID-19 models are needed. To address this, we give many recommendations which, if followed, will solve these issues and lead to higher-quality model development and well-documented manuscripts.
KW - Computational science
KW - Diagnostic markers
KW - Prognostic markers
KW - SARS-Cov-2
UR - http://www.scopus.com/inward/record.url?scp=85103287918&partnerID=8YFLogxK
U2 - 10.1038/s42256-021-00307-0
DO - 10.1038/s42256-021-00307-0
M3 - Article
AN - SCOPUS:85103287918
SN - 2522-5839
VL - 3
SP - 199
EP - 217
JO - Nature Machine Intelligence
JF - Nature Machine Intelligence
IS - 3
ER -