Minimal reporting improvement after peer review in reports of covid-19 prediction models: systematic review

Mohammed T Hudda; Lucinda Archer; Maarten van Smeden; Karel G M Moons; Gary S Collins; Ewout W Steyerberg; Charlotte Wahlich; Johannes B Reitsma; Richard D Riley; Ben Van Calster; Laure Wynants

doi:10.1016/j.jclinepi.2022.12.005

Minimal reporting improvement after peer review in reports of covid-19 prediction models: systematic review

Mohammed T Hudda^*, Lucinda Archer, Maarten van Smeden, Karel G M Moons, Gary S Collins, Ewout W Steyerberg, Charlotte Wahlich, Johannes B Reitsma, Richard D Riley, Ben Van Calster, Laure Wynants

^*Corresponding author for this work

Research output: Contribution to journal › (Systematic) Review article › peer-review

Abstract

OBJECTIVE: To assess improvement in the completeness of reporting COVID-19 prediction models after the peer review process.

STUDY DESIGN AND SETTING: Studies included in a living systematic review of COVID-19 prediction models, with both pre-print and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the TRIPOD reporting guidelines between pre-print and published manuscripts.

RESULTS: 19 studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence amongst pre-print versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from pre-print to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14pp) across all studies. No association was observed between the change in percentage adherence and pre-print score, journal impact factor, or time between journal submission and acceptance.

CONCLUSIONS: Pre-print reporting quality of COVID-19 prediction modelling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.

Original language	English
Pages (from-to)	75-84
Number of pages	10
Journal	Journal of Clinical Epidemiology
Volume	154
Issue number	1
Early online date	14 Dec 2022
DOIs	https://doi.org/10.1016/j.jclinepi.2022.12.005
Publication status	Published - Feb 2023

Access to Document

10.1016/j.jclinepi.2022.12.005Licence: CC BY

Cite this

Hudda, M. T., Archer, L., van Smeden, M., Moons, K. G. M., Collins, G. S., Steyerberg, E. W., Wahlich, C., Reitsma, J. B., Riley, R. D., Van Calster, B., & Wynants, L. (2023). Minimal reporting improvement after peer review in reports of covid-19 prediction models: systematic review. Journal of Clinical Epidemiology, 154(1), 75-84. https://doi.org/10.1016/j.jclinepi.2022.12.005

@article{a5dbadc0979d44ff8e5469db44abe74c,

title = "Minimal reporting improvement after peer review in reports of covid-19 prediction models: systematic review",

abstract = "OBJECTIVE: To assess improvement in the completeness of reporting COVID-19 prediction models after the peer review process.STUDY DESIGN AND SETTING: Studies included in a living systematic review of COVID-19 prediction models, with both pre-print and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the TRIPOD reporting guidelines between pre-print and published manuscripts.RESULTS: 19 studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence amongst pre-print versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from pre-print to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14pp) across all studies. No association was observed between the change in percentage adherence and pre-print score, journal impact factor, or time between journal submission and acceptance.CONCLUSIONS: Pre-print reporting quality of COVID-19 prediction modelling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.",

author = "Hudda, {Mohammed T} and Lucinda Archer and {van Smeden}, Maarten and Moons, {Karel G M} and Collins, {Gary S} and Steyerberg, {Ewout W} and Charlotte Wahlich and Reitsma, {Johannes B} and Riley, {Richard D} and {Van Calster}, Ben and Laure Wynants",

note = "Copyright {\textcopyright} 2022. Published by Elsevier Inc.",

year = "2023",

month = feb,

doi = "10.1016/j.jclinepi.2022.12.005",

language = "English",

volume = "154",

pages = "75--84",

journal = "Journal of Clinical Epidemiology",

issn = "0895-4356",

publisher = "Elsevier Science",

number = "1",

}

TY - JOUR

T1 - Minimal reporting improvement after peer review in reports of covid-19 prediction models

T2 - systematic review

AU - Hudda, Mohammed T

AU - Archer, Lucinda

AU - van Smeden, Maarten

AU - Moons, Karel G M

AU - Collins, Gary S

AU - Steyerberg, Ewout W

AU - Wahlich, Charlotte

AU - Reitsma, Johannes B

AU - Riley, Richard D

AU - Van Calster, Ben

AU - Wynants, Laure

PY - 2023/2

Y1 - 2023/2

N2 - OBJECTIVE: To assess improvement in the completeness of reporting COVID-19 prediction models after the peer review process.STUDY DESIGN AND SETTING: Studies included in a living systematic review of COVID-19 prediction models, with both pre-print and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the TRIPOD reporting guidelines between pre-print and published manuscripts.RESULTS: 19 studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence amongst pre-print versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from pre-print to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14pp) across all studies. No association was observed between the change in percentage adherence and pre-print score, journal impact factor, or time between journal submission and acceptance.CONCLUSIONS: Pre-print reporting quality of COVID-19 prediction modelling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.

AB - OBJECTIVE: To assess improvement in the completeness of reporting COVID-19 prediction models after the peer review process.STUDY DESIGN AND SETTING: Studies included in a living systematic review of COVID-19 prediction models, with both pre-print and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the TRIPOD reporting guidelines between pre-print and published manuscripts.RESULTS: 19 studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence amongst pre-print versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from pre-print to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14pp) across all studies. No association was observed between the change in percentage adherence and pre-print score, journal impact factor, or time between journal submission and acceptance.CONCLUSIONS: Pre-print reporting quality of COVID-19 prediction modelling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.

U2 - 10.1016/j.jclinepi.2022.12.005

DO - 10.1016/j.jclinepi.2022.12.005

M3 - (Systematic) Review article

C2 - 36528232

SN - 0895-4356

VL - 154

SP - 75

EP - 84

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

IS - 1

ER -