The Lung Image Database Consortium, (LIDC) and Image Database Resource Initiative (IDRI):

a completed reference database of lung nodules on CT scans

Samuel G. Armato, Geoffrey McLennan, Luc Bidaut, Michael F. McNitt-Gray, Charles R. Meyer, Anthony P. Reeves, Heber MacMahon, Roger M. Engelmann, Rachael Y. Roberts, Adam Starkey, Philip Caligiuri, Denise R. Aberle, Matthew S. Brown, Richard C. Pais, David P-Y Qing, Poonam Batra, C. Matilda Jude, Iva Petkovska, Alberto M. Biancardi, Binsheng Zhao & 36 others Claudia I. Henschke, David Yankelevitz, Daniel Max, Ali Farooqi, Eric A. Hoffman, Edwin J. R. van Beek, Amanda R. Smith, Ella A. Kazerooni, Peyton H. Bland, Gary E. Laderach, Gregory W. Gladish, Reginald F. Munden, Leslie E. Quint, Lawrence H. Schwartz, Baskaran Sundaram, Lori E. Dodd, Charles Fenimore, David Gur, Nicholas Petrick, John Freymann, Justin Kirby, Brian Hughes, Alessi Vande Casteele, Sangeeta Gupte, Maha Sallam, Michael D. Heath, Michael H. Kuhn, Ekta Dharaiya, Richard Burns, David S. Fryd, Marcos Salganicoff, Vikram Anand, Uri Shreter, Stephen Vastagh, Barbara Y. Croft, Laurence P. Clarke

    Research output: Contribution to journalArticle

    541 Citations (Scopus)

    Abstract

    Purpose: The development of computer-aided diagnostic (CAD) methods for lung nodule detection, classification, and quantitative assessment can be facilitated through a well-characterized repository of computed tomography (CT) scans. The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI) completed such a database, establishing a publicly available reference for the medical imaging research community. Initiated by the National Cancer Institute (NCI), further advanced by the Foundation for the National Institutes of Health (FNIH), and accompanied by the Food and Drug Administration (FDA) through active participation, this public-private partnership demonstrates the success of a consortium founded on a consensus-based process.

    Methods: Seven academic centers and eight medical imaging companies collaborated to identify, address, and resolve challenging organizational, technical, and clinical issues to provide a solid foundation for a robust database. The LIDC/IDRI Database contains 1018 cases, each of which includes images from a clinical thoracic CT scan and an associated XML file that records the results of a two-phase image annotation process performed by four experienced thoracic radiologists. In the initial blinded-read phase, each radiologist independently reviewed each CT scan and marked lesions belonging to one of three categories ("nodule >= 3 mm," "nodule < 3 mm," and "non-nodule >= 3 mm"). In the subsequent unblinded-read phase, each radiologist independently reviewed their own marks along with the anonymized marks of the three other radiologists to render a final opinion. The goal of this process was to identify as completely as possible all lung nodules in each CT scan without requiring forced consensus.

    Results: The Database contains 7371 lesions marked "nodule" by at least one radiologist. 2669 of these lesions were marked "nodule >= 3 mm" by at least one radiologist, of which 928 (34.7%) received such marks from all four radiologists. These 2669 lesions include nodule outlines and subjective nodule characteristic ratings.

    Conclusions: The LIDC/IDRI Database is expected to provide an essential medical imaging research resource to spur CAD development, validation, and dissemination in clinical practice. [DOI: 10.1118/1.3528204]

    Original languageEnglish
    Pages (from-to)915-931
    Number of pages17
    JournalMedical Physics
    Volume38
    Issue number2
    Early online date24 Jan 2011
    DOIs
    Publication statusPublished - Feb 2011

    Keywords

    • Lung nodule
    • Computed tomography (CT)
    • Thoracic imaging
    • Interobserver variability
    • Computer-aided diagnosis (CAD)
    • Pulmonary nodules
    • Screening trial
    • Aided detection
    • Cancer
    • Truth
    • Performance
    • Selection
    • Nelson
    • CAD

    Cite this

    Armato, S. G., McLennan, G., Bidaut, L., McNitt-Gray, M. F., Meyer, C. R., Reeves, A. P., ... Clarke, L. P. (2011). The Lung Image Database Consortium, (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. Medical Physics, 38(2), 915-931. https://doi.org/10.1118/1.3528204
    Armato, Samuel G. ; McLennan, Geoffrey ; Bidaut, Luc ; McNitt-Gray, Michael F. ; Meyer, Charles R. ; Reeves, Anthony P. ; MacMahon, Heber ; Engelmann, Roger M. ; Roberts, Rachael Y. ; Starkey, Adam ; Caligiuri, Philip ; Aberle, Denise R. ; Brown, Matthew S. ; Pais, Richard C. ; Qing, David P-Y ; Batra, Poonam ; Jude, C. Matilda ; Petkovska, Iva ; Biancardi, Alberto M. ; Zhao, Binsheng ; Henschke, Claudia I. ; Yankelevitz, David ; Max, Daniel ; Farooqi, Ali ; Hoffman, Eric A. ; van Beek, Edwin J. R. ; Smith, Amanda R. ; Kazerooni, Ella A. ; Bland, Peyton H. ; Laderach, Gary E. ; Gladish, Gregory W. ; Munden, Reginald F. ; Quint, Leslie E. ; Schwartz, Lawrence H. ; Sundaram, Baskaran ; Dodd, Lori E. ; Fenimore, Charles ; Gur, David ; Petrick, Nicholas ; Freymann, John ; Kirby, Justin ; Hughes, Brian ; Casteele, Alessi Vande ; Gupte, Sangeeta ; Sallam, Maha ; Heath, Michael D. ; Kuhn, Michael H. ; Dharaiya, Ekta ; Burns, Richard ; Fryd, David S. ; Salganicoff, Marcos ; Anand, Vikram ; Shreter, Uri ; Vastagh, Stephen ; Croft, Barbara Y. ; Clarke, Laurence P. / The Lung Image Database Consortium, (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. In: Medical Physics. 2011 ; Vol. 38, No. 2. pp. 915-931.
    @article{a4d01420decb47e5ac928fa63a199de0,
    title = "The Lung Image Database Consortium, (LIDC) and Image Database Resource Initiative (IDRI):: a completed reference database of lung nodules on CT scans",
    abstract = "Purpose: The development of computer-aided diagnostic (CAD) methods for lung nodule detection, classification, and quantitative assessment can be facilitated through a well-characterized repository of computed tomography (CT) scans. The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI) completed such a database, establishing a publicly available reference for the medical imaging research community. Initiated by the National Cancer Institute (NCI), further advanced by the Foundation for the National Institutes of Health (FNIH), and accompanied by the Food and Drug Administration (FDA) through active participation, this public-private partnership demonstrates the success of a consortium founded on a consensus-based process.Methods: Seven academic centers and eight medical imaging companies collaborated to identify, address, and resolve challenging organizational, technical, and clinical issues to provide a solid foundation for a robust database. The LIDC/IDRI Database contains 1018 cases, each of which includes images from a clinical thoracic CT scan and an associated XML file that records the results of a two-phase image annotation process performed by four experienced thoracic radiologists. In the initial blinded-read phase, each radiologist independently reviewed each CT scan and marked lesions belonging to one of three categories ({"}nodule >= 3 mm,{"} {"}nodule < 3 mm,{"} and {"}non-nodule >= 3 mm{"}). In the subsequent unblinded-read phase, each radiologist independently reviewed their own marks along with the anonymized marks of the three other radiologists to render a final opinion. The goal of this process was to identify as completely as possible all lung nodules in each CT scan without requiring forced consensus.Results: The Database contains 7371 lesions marked {"}nodule{"} by at least one radiologist. 2669 of these lesions were marked {"}nodule >= 3 mm{"} by at least one radiologist, of which 928 (34.7{\%}) received such marks from all four radiologists. These 2669 lesions include nodule outlines and subjective nodule characteristic ratings.Conclusions: The LIDC/IDRI Database is expected to provide an essential medical imaging research resource to spur CAD development, validation, and dissemination in clinical practice. [DOI: 10.1118/1.3528204]",
    keywords = "Lung nodule, Computed tomography (CT), Thoracic imaging, Interobserver variability, Computer-aided diagnosis (CAD), Pulmonary nodules, Screening trial, Aided detection, Cancer, Truth, Performance, Selection, Nelson, CAD",
    author = "Armato, {Samuel G.} and Geoffrey McLennan and Luc Bidaut and McNitt-Gray, {Michael F.} and Meyer, {Charles R.} and Reeves, {Anthony P.} and Heber MacMahon and Engelmann, {Roger M.} and Roberts, {Rachael Y.} and Adam Starkey and Philip Caligiuri and Aberle, {Denise R.} and Brown, {Matthew S.} and Pais, {Richard C.} and Qing, {David P-Y} and Poonam Batra and Jude, {C. Matilda} and Iva Petkovska and Biancardi, {Alberto M.} and Binsheng Zhao and Henschke, {Claudia I.} and David Yankelevitz and Daniel Max and Ali Farooqi and Hoffman, {Eric A.} and {van Beek}, {Edwin J. R.} and Smith, {Amanda R.} and Kazerooni, {Ella A.} and Bland, {Peyton H.} and Laderach, {Gary E.} and Gladish, {Gregory W.} and Munden, {Reginald F.} and Quint, {Leslie E.} and Schwartz, {Lawrence H.} and Baskaran Sundaram and Dodd, {Lori E.} and Charles Fenimore and David Gur and Nicholas Petrick and John Freymann and Justin Kirby and Brian Hughes and Casteele, {Alessi Vande} and Sangeeta Gupte and Maha Sallam and Heath, {Michael D.} and Kuhn, {Michael H.} and Ekta Dharaiya and Richard Burns and Fryd, {David S.} and Marcos Salganicoff and Vikram Anand and Uri Shreter and Stephen Vastagh and Croft, {Barbara Y.} and Clarke, {Laurence P.}",
    year = "2011",
    month = "2",
    doi = "10.1118/1.3528204",
    language = "English",
    volume = "38",
    pages = "915--931",
    journal = "Medical Physics",
    issn = "0094-2405",
    publisher = "American Association of Physicists in Medicine",
    number = "2",

    }

    Armato, SG, McLennan, G, Bidaut, L, McNitt-Gray, MF, Meyer, CR, Reeves, AP, MacMahon, H, Engelmann, RM, Roberts, RY, Starkey, A, Caligiuri, P, Aberle, DR, Brown, MS, Pais, RC, Qing, DP-Y, Batra, P, Jude, CM, Petkovska, I, Biancardi, AM, Zhao, B, Henschke, CI, Yankelevitz, D, Max, D, Farooqi, A, Hoffman, EA, van Beek, EJR, Smith, AR, Kazerooni, EA, Bland, PH, Laderach, GE, Gladish, GW, Munden, RF, Quint, LE, Schwartz, LH, Sundaram, B, Dodd, LE, Fenimore, C, Gur, D, Petrick, N, Freymann, J, Kirby, J, Hughes, B, Casteele, AV, Gupte, S, Sallam, M, Heath, MD, Kuhn, MH, Dharaiya, E, Burns, R, Fryd, DS, Salganicoff, M, Anand, V, Shreter, U, Vastagh, S, Croft, BY & Clarke, LP 2011, 'The Lung Image Database Consortium, (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans', Medical Physics, vol. 38, no. 2, pp. 915-931. https://doi.org/10.1118/1.3528204

    The Lung Image Database Consortium, (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. / Armato, Samuel G.; McLennan, Geoffrey; Bidaut, Luc; McNitt-Gray, Michael F.; Meyer, Charles R.; Reeves, Anthony P.; MacMahon, Heber; Engelmann, Roger M.; Roberts, Rachael Y.; Starkey, Adam; Caligiuri, Philip; Aberle, Denise R.; Brown, Matthew S.; Pais, Richard C.; Qing, David P-Y; Batra, Poonam; Jude, C. Matilda; Petkovska, Iva; Biancardi, Alberto M.; Zhao, Binsheng; Henschke, Claudia I.; Yankelevitz, David; Max, Daniel; Farooqi, Ali; Hoffman, Eric A.; van Beek, Edwin J. R.; Smith, Amanda R.; Kazerooni, Ella A.; Bland, Peyton H.; Laderach, Gary E.; Gladish, Gregory W.; Munden, Reginald F.; Quint, Leslie E.; Schwartz, Lawrence H.; Sundaram, Baskaran; Dodd, Lori E.; Fenimore, Charles; Gur, David; Petrick, Nicholas; Freymann, John; Kirby, Justin; Hughes, Brian; Casteele, Alessi Vande; Gupte, Sangeeta; Sallam, Maha; Heath, Michael D.; Kuhn, Michael H.; Dharaiya, Ekta; Burns, Richard; Fryd, David S.; Salganicoff, Marcos; Anand, Vikram; Shreter, Uri; Vastagh, Stephen; Croft, Barbara Y.; Clarke, Laurence P.

    In: Medical Physics, Vol. 38, No. 2, 02.2011, p. 915-931.

    Research output: Contribution to journalArticle

    TY - JOUR

    T1 - The Lung Image Database Consortium, (LIDC) and Image Database Resource Initiative (IDRI):

    T2 - a completed reference database of lung nodules on CT scans

    AU - Armato, Samuel G.

    AU - McLennan, Geoffrey

    AU - Bidaut, Luc

    AU - McNitt-Gray, Michael F.

    AU - Meyer, Charles R.

    AU - Reeves, Anthony P.

    AU - MacMahon, Heber

    AU - Engelmann, Roger M.

    AU - Roberts, Rachael Y.

    AU - Starkey, Adam

    AU - Caligiuri, Philip

    AU - Aberle, Denise R.

    AU - Brown, Matthew S.

    AU - Pais, Richard C.

    AU - Qing, David P-Y

    AU - Batra, Poonam

    AU - Jude, C. Matilda

    AU - Petkovska, Iva

    AU - Biancardi, Alberto M.

    AU - Zhao, Binsheng

    AU - Henschke, Claudia I.

    AU - Yankelevitz, David

    AU - Max, Daniel

    AU - Farooqi, Ali

    AU - Hoffman, Eric A.

    AU - van Beek, Edwin J. R.

    AU - Smith, Amanda R.

    AU - Kazerooni, Ella A.

    AU - Bland, Peyton H.

    AU - Laderach, Gary E.

    AU - Gladish, Gregory W.

    AU - Munden, Reginald F.

    AU - Quint, Leslie E.

    AU - Schwartz, Lawrence H.

    AU - Sundaram, Baskaran

    AU - Dodd, Lori E.

    AU - Fenimore, Charles

    AU - Gur, David

    AU - Petrick, Nicholas

    AU - Freymann, John

    AU - Kirby, Justin

    AU - Hughes, Brian

    AU - Casteele, Alessi Vande

    AU - Gupte, Sangeeta

    AU - Sallam, Maha

    AU - Heath, Michael D.

    AU - Kuhn, Michael H.

    AU - Dharaiya, Ekta

    AU - Burns, Richard

    AU - Fryd, David S.

    AU - Salganicoff, Marcos

    AU - Anand, Vikram

    AU - Shreter, Uri

    AU - Vastagh, Stephen

    AU - Croft, Barbara Y.

    AU - Clarke, Laurence P.

    PY - 2011/2

    Y1 - 2011/2

    N2 - Purpose: The development of computer-aided diagnostic (CAD) methods for lung nodule detection, classification, and quantitative assessment can be facilitated through a well-characterized repository of computed tomography (CT) scans. The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI) completed such a database, establishing a publicly available reference for the medical imaging research community. Initiated by the National Cancer Institute (NCI), further advanced by the Foundation for the National Institutes of Health (FNIH), and accompanied by the Food and Drug Administration (FDA) through active participation, this public-private partnership demonstrates the success of a consortium founded on a consensus-based process.Methods: Seven academic centers and eight medical imaging companies collaborated to identify, address, and resolve challenging organizational, technical, and clinical issues to provide a solid foundation for a robust database. The LIDC/IDRI Database contains 1018 cases, each of which includes images from a clinical thoracic CT scan and an associated XML file that records the results of a two-phase image annotation process performed by four experienced thoracic radiologists. In the initial blinded-read phase, each radiologist independently reviewed each CT scan and marked lesions belonging to one of three categories ("nodule >= 3 mm," "nodule < 3 mm," and "non-nodule >= 3 mm"). In the subsequent unblinded-read phase, each radiologist independently reviewed their own marks along with the anonymized marks of the three other radiologists to render a final opinion. The goal of this process was to identify as completely as possible all lung nodules in each CT scan without requiring forced consensus.Results: The Database contains 7371 lesions marked "nodule" by at least one radiologist. 2669 of these lesions were marked "nodule >= 3 mm" by at least one radiologist, of which 928 (34.7%) received such marks from all four radiologists. These 2669 lesions include nodule outlines and subjective nodule characteristic ratings.Conclusions: The LIDC/IDRI Database is expected to provide an essential medical imaging research resource to spur CAD development, validation, and dissemination in clinical practice. [DOI: 10.1118/1.3528204]

    AB - Purpose: The development of computer-aided diagnostic (CAD) methods for lung nodule detection, classification, and quantitative assessment can be facilitated through a well-characterized repository of computed tomography (CT) scans. The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI) completed such a database, establishing a publicly available reference for the medical imaging research community. Initiated by the National Cancer Institute (NCI), further advanced by the Foundation for the National Institutes of Health (FNIH), and accompanied by the Food and Drug Administration (FDA) through active participation, this public-private partnership demonstrates the success of a consortium founded on a consensus-based process.Methods: Seven academic centers and eight medical imaging companies collaborated to identify, address, and resolve challenging organizational, technical, and clinical issues to provide a solid foundation for a robust database. The LIDC/IDRI Database contains 1018 cases, each of which includes images from a clinical thoracic CT scan and an associated XML file that records the results of a two-phase image annotation process performed by four experienced thoracic radiologists. In the initial blinded-read phase, each radiologist independently reviewed each CT scan and marked lesions belonging to one of three categories ("nodule >= 3 mm," "nodule < 3 mm," and "non-nodule >= 3 mm"). In the subsequent unblinded-read phase, each radiologist independently reviewed their own marks along with the anonymized marks of the three other radiologists to render a final opinion. The goal of this process was to identify as completely as possible all lung nodules in each CT scan without requiring forced consensus.Results: The Database contains 7371 lesions marked "nodule" by at least one radiologist. 2669 of these lesions were marked "nodule >= 3 mm" by at least one radiologist, of which 928 (34.7%) received such marks from all four radiologists. These 2669 lesions include nodule outlines and subjective nodule characteristic ratings.Conclusions: The LIDC/IDRI Database is expected to provide an essential medical imaging research resource to spur CAD development, validation, and dissemination in clinical practice. [DOI: 10.1118/1.3528204]

    KW - Lung nodule

    KW - Computed tomography (CT)

    KW - Thoracic imaging

    KW - Interobserver variability

    KW - Computer-aided diagnosis (CAD)

    KW - Pulmonary nodules

    KW - Screening trial

    KW - Aided detection

    KW - Cancer

    KW - Truth

    KW - Performance

    KW - Selection

    KW - Nelson

    KW - CAD

    U2 - 10.1118/1.3528204

    DO - 10.1118/1.3528204

    M3 - Article

    VL - 38

    SP - 915

    EP - 931

    JO - Medical Physics

    JF - Medical Physics

    SN - 0094-2405

    IS - 2

    ER -