Minimal reporting improvement after peer review in reports of covid-19 prediction models: systematic review

Mohammed T Hudda; Lucinda Archer; Maarten van Smeden; Karel G M Moons; Gary S Collins; Ewout W Steyerberg; Charlotte Wahlich; Johannes B Reitsma; Richard D Riley; Ben Van Calster; Laure Wynants

doi:10.1016/j.jclinepi.2022.12.005

Minimal reporting improvement after peer review in reports of covid-19 prediction models: systematic review

Mohammed T Hudda^*, Lucinda Archer, Maarten van Smeden, Karel G M Moons, Gary S Collins, Ewout W Steyerberg, Charlotte Wahlich, Johannes B Reitsma, Richard D Riley, Ben Van Calster, Laure Wynants

^*Corresponding author for this work

Research output: Contribution to journal › Article › Academic › peer-review

Abstract

OBJECTIVE: To assess improvement in the completeness of reporting COVID-19 prediction models after the peer review process.

STUDY DESIGN AND SETTING: Studies included in a living systematic review of COVID-19 prediction models, with both pre-print and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the TRIPOD reporting guidelines between pre-print and published manuscripts.

RESULTS: 19 studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence amongst pre-print versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from pre-print to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14pp) across all studies. No association was observed between the change in percentage adherence and pre-print score, journal impact factor, or time between journal submission and acceptance.

CONCLUSIONS: Pre-print reporting quality of COVID-19 prediction modelling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.

Original language	English
Journal	Journal of Clinical Epidemiology
DOIs	https://doi.org/10.1016/j.jclinepi.2022.12.005
Publication status	E-pub ahead of print - 14 Dec 2022

Access to Document

10.1016/j.jclinepi.2022.12.005

Cite this

@article{a5dbadc0979d44ff8e5469db44abe74c,

title = "Minimal reporting improvement after peer review in reports of covid-19 prediction models: systematic review",

abstract = "OBJECTIVE: To assess improvement in the completeness of reporting COVID-19 prediction models after the peer review process.STUDY DESIGN AND SETTING: Studies included in a living systematic review of COVID-19 prediction models, with both pre-print and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the TRIPOD reporting guidelines between pre-print and published manuscripts.RESULTS: 19 studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence amongst pre-print versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from pre-print to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14pp) across all studies. No association was observed between the change in percentage adherence and pre-print score, journal impact factor, or time between journal submission and acceptance.CONCLUSIONS: Pre-print reporting quality of COVID-19 prediction modelling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.",

author = "Hudda, {Mohammed T} and Lucinda Archer and {van Smeden}, Maarten and Moons, {Karel G M} and Collins, {Gary S} and Steyerberg, {Ewout W} and Charlotte Wahlich and Reitsma, {Johannes B} and Riley, {Richard D} and {Van Calster}, Ben and Laure Wynants",

note = "Copyright {\textcopyright} 2022. Published by Elsevier Inc.",

year = "2022",

month = dec,

day = "14",

doi = "10.1016/j.jclinepi.2022.12.005",

language = "English",

journal = "Journal of Clinical Epidemiology",

issn = "0895-4356",

publisher = "Elsevier Science",

}

TY - JOUR

T1 - Minimal reporting improvement after peer review in reports of covid-19 prediction models

T2 - systematic review

AU - Hudda, Mohammed T

AU - Archer, Lucinda

AU - van Smeden, Maarten

AU - Moons, Karel G M

AU - Collins, Gary S

AU - Steyerberg, Ewout W

AU - Wahlich, Charlotte

AU - Reitsma, Johannes B

AU - Riley, Richard D

AU - Van Calster, Ben

AU - Wynants, Laure

PY - 2022/12/14

Y1 - 2022/12/14

N2 - OBJECTIVE: To assess improvement in the completeness of reporting COVID-19 prediction models after the peer review process.STUDY DESIGN AND SETTING: Studies included in a living systematic review of COVID-19 prediction models, with both pre-print and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the TRIPOD reporting guidelines between pre-print and published manuscripts.RESULTS: 19 studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence amongst pre-print versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from pre-print to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14pp) across all studies. No association was observed between the change in percentage adherence and pre-print score, journal impact factor, or time between journal submission and acceptance.CONCLUSIONS: Pre-print reporting quality of COVID-19 prediction modelling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.

AB - OBJECTIVE: To assess improvement in the completeness of reporting COVID-19 prediction models after the peer review process.STUDY DESIGN AND SETTING: Studies included in a living systematic review of COVID-19 prediction models, with both pre-print and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the TRIPOD reporting guidelines between pre-print and published manuscripts.RESULTS: 19 studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence amongst pre-print versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from pre-print to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14pp) across all studies. No association was observed between the change in percentage adherence and pre-print score, journal impact factor, or time between journal submission and acceptance.CONCLUSIONS: Pre-print reporting quality of COVID-19 prediction modelling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.

U2 - 10.1016/j.jclinepi.2022.12.005

DO - 10.1016/j.jclinepi.2022.12.005

M3 - Article

C2 - 36528232

SN - 0895-4356

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

ER -