Automated Assessment of COVID-19 Reporting and Data System and Chest CT Severity Scores in Patients Suspected of Having COVID-19 Using Artificial Intelligence

Nikolas Lessmann, Clara Sanchez, Ludo Beenen, Luuk H. Boulogne, Monique Brink, Erdi Calli, Jean-Paul Charbonnier, Ton Dofferhoff, Wouter M. van Everdingen, Paul K. Gerke, Bram Geurts, Hester A. Gietema, Miriam Groeneveld, Louis van Harten, Nils Hendrix, Ward Hendrix, Henkjan J. Huisman, Ivana Isgum, Colin Jacobs, Ruben KlugeMichel Kok, Jasenko Krdzalic, Bianca Lassen-Schmidt, Kicky van Leeuwen, James Meakin, Mike Overkamp, Tjalco van Rees Vellinga, Eva M. van Rikxoort, Riccardo Samperna, Cornelia Schaefer-Prokop, Steven Schalekamp, Ernst Th Scholten, Cheryl Sital, J. Lauran Stoeger, Jonas Teuwen, Kiran Vaidhya Venkadesh, Coen de Vente, Marieke Vermaat, Weiyi Xie, Bram de Wilde, Mathias Prokop, Bram van Ginneken*

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review


Background: The coronavirus disease 2019 (COVID-19) pandemic has spread across the globe with alarming speed, morbidity, and mortality. Immediate triage of patients with chest infections suspected to be caused by COVID-19 using chest CT may be of assistance when results from definitive viral testing are delayed.

Purpose: To develop and validate an artificial intelligence (AI) system to score the likelihood and extent of pulmonary COVID-19 on chest CT scans using the COVID-19 Reporting and Data System (CO-RADS) and CT severity scoring systems.

Materials and Methods: The CO-RADS AI system consists of three deep-learning algorithms that automatically segment the five pulmonary lobes, assign a CO-RADS score for the suspicion of COVID-19, and assign a CT severity score for the degree of parenchymal involvement per lobe. This study retrospectively included patients who underwent a nonenhanced chest CT examination because of clinical suspicion of COVID-19 at two medical centers. The system was trained, validated, and tested with data from one of the centers. Data from the second center served as an external test set. Diagnostic performance and agreement with scores assigned by eight independent observers were measured using receiver operating characteristic analysis, linearly weighted kappa values, and classification accuracy.

Results: A total of 105 patients (mean age, 62 years +/- 16 [standard deviation]; 61 men) and 262 patients (mean age, 64 years +/- 16; 154 men) were evaluated in the internal and external test sets, respectively. The system discriminated between patients with COVID-19 and those without COVID-19, with areas under the receiver operating characteristic curve of 0.95 (95% CI: 0.91, 0.98) and 0.88 (95% CI: 0.84, 0.93), for the internal and external test sets, respectively. Agreement with the eight human observers was moderate to substantial, with mean linearly weighted k values of 0.60 +/- 0.01 for CO-RADS scores and 0.54 +/- 0.01 for CT severity scores.

Conclusion: With high diagnostic performance, the CO-RADS AI system correctly identified patients with COVID-19 using chest CT scans and assigned standardized CO-RADS and CT severity scores that demonstrated good agreement with findings from eight independent observers and generalized well to external data. (C) RSNA, 2020

Original languageEnglish
Pages (from-to)E18-E28
Number of pages11
Issue number1
Publication statusPublished - Jan 2021




Dive into the research topics of 'Automated Assessment of COVID-19 Reporting and Data System and Chest CT Severity Scores in Patients Suspected of Having COVID-19 Using Artificial Intelligence'. Together they form a unique fingerprint.

Cite this