Predicting COVID-19 prognosis in the ICU remained challenging: external validation in a multinational regional cohort

Daniek A M Meijs*, Sander M J van Kuijk, Laure Wynants, Björn Stessel, Jannet Mehagnoul-Schipper, Anisa Hana, Clarissa I E Scheeren, Dennis C J J Bergmans, Johannes Bickenbach, Margot Vander Laenen, Luc J M Smits, Iwan C C van der Horst, Gernot Marx, Dieter Mesotten, Bas C T van Bussel, CoDaP Investigators, Mark Mulder, Julia Bels, Nick Wilmes, Emma JanssenMicheline Florack, Chahinda Ghossein

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

OBJECTIVES: Many prediction models for coronavirus disease 2019 (COVID-19) have been developed. External validation is mandatory before implementation in the intensive care unit (ICU). We selected and validated prognostic models in the Euregio Intensive Care COVID (EICC) cohort.

STUDY DESIGN AND SETTING: In this multinational cohort study, routine data from COVID-19 patients admitted to ICUs within the Euregio Meuse-Rhine were collected from March to August 2020. COVID-19 models were selected based on model type, predictors, outcomes, and reporting. Furthermore, general ICU scores were assessed. Discrimination was assessed by area under the receiver operating characteristic curves (AUCs) and calibration by calibration-in-the-large and calibration plots. A random-effects meta-analysis was used to pool results.

RESULTS: 551 patients were admitted. Mean age was 65.4 ± 11.2 years, 29% were female, and ICU mortality was 36%. Nine out of 238 published models were externally validated. Pooled AUCs were between 0.53 and 0.70 and calibration-in-the-large between -9% and 6%. Calibration plots showed generally poor but, for the 4C Mortality score and Spanish Society of Infectious Diseases and Clinical Microbiology (SEIMC) score, moderate calibration.

CONCLUSION: Of the nine prognostic models that were externally validated in the EICC cohort, only two showed reasonable discrimination and moderate calibration. For future pandemics, better models based on routine data are needed to support admission decision-making.

Original languageEnglish
Pages (from-to)257-268
Number of pages12
JournalJournal of Clinical Epidemiology
Volume152
Early online date27 Oct 2022
DOIs
Publication statusPublished - Dec 2022

Cite this