PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies

Robert F. Wolff; Karel G. M. Moons; Richard D. Riley; Penny F. Whiting; Marie Westwood; Gary S. Collins; Johannes B. Reitsma; Jos Kleijnen; Sue Mallett; Doug Altman; Patrick Bossuyt; Nancy R. Cook; Gennaro D'Amico; Thomas P. A. Debray; Jon Deeks; Joris de Groot; Emanuele di Angelantonio; Tom Fahey; Frank Harrell; Jill A. Hayden; Martijn W. Heymans; Lotty Hooft; Chris Hyde; John Ioannidis; Alfonso Iorio; Stephen Kaptoge; Andre Knottnerus; Mariska Leeflang; Frances Nixon; Pablo Perel; Bob Phillips; Heike Raatz; Rob Riemsma; Maroeska Rovers; Anne W. S. Rutjes; Willi Sauerbrei; Stefan Sauerland; Fueloep Scheibler; Rob Scholten; Ewoud Schuit; Ewout Steyerberg; Toni Tan; Gerben ter Riet; Danielle van der Windt; Yvonne Vergouwe; Andrew Vickers; Angela M. Wood; PROBAST group; PROBAST Steering Group; PROBAST Delphi Group

doi:10.7326/M18-1376

PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies

Robert F. Wolff^*, Karel G. M. Moons, Richard D. Riley, Penny F. Whiting, Marie Westwood, Gary S. Collins, Johannes B. Reitsma, Jos Kleijnen, Sue Mallett, Doug Altman, Patrick Bossuyt, Nancy R. Cook, Gennaro D'Amico, Thomas P. A. Debray, Jon Deeks, Joris de Groot, Emanuele di Angelantonio, Tom Fahey, Frank Harrell, Jill A. HaydenMartijn W. Heymans, Lotty Hooft, Chris Hyde, John Ioannidis, Alfonso Iorio, Stephen Kaptoge, Andre Knottnerus, Mariska Leeflang, Frances Nixon, Pablo Perel, Bob Phillips, Heike Raatz, Rob Riemsma, Maroeska Rovers, Anne W. S. Rutjes, Willi Sauerbrei, Stefan Sauerland, Fueloep Scheibler, Rob Scholten, Ewoud Schuit, Ewout Steyerberg, Toni Tan, Gerben ter Riet, Danielle van der Windt, Yvonne Vergouwe, Andrew Vickers, Angela M. Wood, PROBAST group, PROBAST Steering Group, PROBAST Delphi Group

^*Corresponding author for this work

Research output: Contribution to journal › Article › Academic › peer-review

572 Citations (Web of Science)

Abstract

Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models).

PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was developed by a steering group that considered existing ROB tools and reporting guidelines. The tool was informed by a Delphi procedure involving 38 experts and was refined through piloting.

PROBAST is organized into the following 4 domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signaling questions to facilitate structured judgment of ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions.

Although PROBAST was designed for systematic reviews, it can be used more generally in critical appraisal of prediction model studies. Potential users include organizations supporting decision making, researchers and clinicians who are interested in evidence-based medicine or involved in guideline development, journal editors, and manuscript reviewers.

Original language	English
Pages (from-to)	51-60
Number of pages	10
Journal	Annals of Internal Medicine
Volume	170
Issue number	1
DOIs	https://doi.org/10.7326/M18-1376
Publication status	Published - 1 Jan 2019

Keywords

DIAGNOSIS TRIPOD
EXPLANATION
INDIVIDUAL PROGNOSIS
CARDIOVASCULAR RISK

Access to Document

10.7326/M18-1376

Cite this

Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., Altman, D., Bossuyt, P., Cook, N. R., D'Amico, G., Debray, T. P. A., Deeks, J., de Groot, J., di Angelantonio, E., Fahey, T., Harrell, F., ... PROBAST Delphi Group (2019). PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies. Annals of Internal Medicine, 170(1), 51-60. https://doi.org/10.7326/M18-1376

@article{c897360e438a4d508717d4d0da5136fa,

title = "PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies",

abstract = "Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models).PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was developed by a steering group that considered existing ROB tools and reporting guidelines. The tool was informed by a Delphi procedure involving 38 experts and was refined through piloting.PROBAST is organized into the following 4 domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signaling questions to facilitate structured judgment of ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions.Although PROBAST was designed for systematic reviews, it can be used more generally in critical appraisal of prediction model studies. Potential users include organizations supporting decision making, researchers and clinicians who are interested in evidence-based medicine or involved in guideline development, journal editors, and manuscript reviewers.",

keywords = "DIAGNOSIS TRIPOD, EXPLANATION, INDIVIDUAL PROGNOSIS, CARDIOVASCULAR RISK",

author = "Wolff, {Robert F.} and Moons, {Karel G. M.} and Riley, {Richard D.} and Whiting, {Penny F.} and Marie Westwood and Collins, {Gary S.} and Reitsma, {Johannes B.} and Jos Kleijnen and Sue Mallett and Doug Altman and Patrick Bossuyt and Cook, {Nancy R.} and Gennaro D'Amico and Debray, {Thomas P. A.} and Jon Deeks and {de Groot}, Joris and {di Angelantonio}, Emanuele and Tom Fahey and Frank Harrell and Hayden, {Jill A.} and Heymans, {Martijn W.} and Lotty Hooft and Chris Hyde and John Ioannidis and Alfonso Iorio and Stephen Kaptoge and Andre Knottnerus and Mariska Leeflang and Frances Nixon and Pablo Perel and Bob Phillips and Heike Raatz and Rob Riemsma and Maroeska Rovers and Rutjes, {Anne W. S.} and Willi Sauerbrei and Stefan Sauerland and Fueloep Scheibler and Rob Scholten and Ewoud Schuit and Ewout Steyerberg and Toni Tan and {ter Riet}, Gerben and {van der Windt}, Danielle and Yvonne Vergouwe and Andrew Vickers and Wood, {Angela M.} and {PROBAST group} and {PROBAST Steering Group} and {PROBAST Delphi Group}",

year = "2019",

month = jan,

day = "1",

doi = "10.7326/M18-1376",

language = "English",

volume = "170",

pages = "51--60",

journal = "Annals of Internal Medicine",

issn = "0003-4819",

publisher = "American College of Physicians",

number = "1",

}

Wolff, RF, Moons, KGM, Riley, RD, Whiting, PF, Westwood, M, Collins, GS, Reitsma, JB, Kleijnen, J, Mallett, S, Altman, D, Bossuyt, P, Cook, NR, D'Amico, G, Debray, TPA, Deeks, J, de Groot, J, di Angelantonio, E, Fahey, T, Harrell, F, Hayden, JA, Heymans, MW, Hooft, L, Hyde, C, Ioannidis, J, Iorio, A, Kaptoge, S, Knottnerus, A, Leeflang, M, Nixon, F, Perel, P, Phillips, B, Raatz, H, Riemsma, R, Rovers, M, Rutjes, AWS, Sauerbrei, W, Sauerland, S, Scheibler, F, Scholten, R, Schuit, E, Steyerberg, E, Tan, T, ter Riet, G, van der Windt, D, Vergouwe, Y, Vickers, A, Wood, AM, PROBAST group, PROBAST Steering Group & PROBAST Delphi Group 2019, 'PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies', Annals of Internal Medicine, vol. 170, no. 1, pp. 51-60. https://doi.org/10.7326/M18-1376

TY - JOUR

T1 - PROBAST

T2 - A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies

AU - Wolff, Robert F.

AU - Moons, Karel G. M.

AU - Riley, Richard D.

AU - Whiting, Penny F.

AU - Westwood, Marie

AU - Collins, Gary S.

AU - Reitsma, Johannes B.

AU - Kleijnen, Jos

AU - Mallett, Sue

AU - Altman, Doug

AU - Bossuyt, Patrick

AU - Cook, Nancy R.

AU - D'Amico, Gennaro

AU - Debray, Thomas P. A.

AU - Deeks, Jon

AU - de Groot, Joris

AU - di Angelantonio, Emanuele

AU - Fahey, Tom

AU - Harrell, Frank

AU - Hayden, Jill A.

AU - Heymans, Martijn W.

AU - Hooft, Lotty

AU - Hyde, Chris

AU - Ioannidis, John

AU - Iorio, Alfonso

AU - Kaptoge, Stephen

AU - Knottnerus, Andre

AU - Leeflang, Mariska

AU - Nixon, Frances

AU - Perel, Pablo

AU - Phillips, Bob

AU - Raatz, Heike

AU - Riemsma, Rob

AU - Rovers, Maroeska

AU - Rutjes, Anne W. S.

AU - Sauerbrei, Willi

AU - Sauerland, Stefan

AU - Scheibler, Fueloep

AU - Scholten, Rob

AU - Schuit, Ewoud

AU - Steyerberg, Ewout

AU - Tan, Toni

AU - ter Riet, Gerben

AU - van der Windt, Danielle

AU - Vergouwe, Yvonne

AU - Vickers, Andrew

AU - Wood, Angela M.

AU - PROBAST group

AU - PROBAST Steering Group

AU - PROBAST Delphi Group

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models).PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was developed by a steering group that considered existing ROB tools and reporting guidelines. The tool was informed by a Delphi procedure involving 38 experts and was refined through piloting.PROBAST is organized into the following 4 domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signaling questions to facilitate structured judgment of ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions.Although PROBAST was designed for systematic reviews, it can be used more generally in critical appraisal of prediction model studies. Potential users include organizations supporting decision making, researchers and clinicians who are interested in evidence-based medicine or involved in guideline development, journal editors, and manuscript reviewers.

AB - Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models).PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was developed by a steering group that considered existing ROB tools and reporting guidelines. The tool was informed by a Delphi procedure involving 38 experts and was refined through piloting.PROBAST is organized into the following 4 domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signaling questions to facilitate structured judgment of ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions.Although PROBAST was designed for systematic reviews, it can be used more generally in critical appraisal of prediction model studies. Potential users include organizations supporting decision making, researchers and clinicians who are interested in evidence-based medicine or involved in guideline development, journal editors, and manuscript reviewers.

KW - DIAGNOSIS TRIPOD

KW - EXPLANATION

KW - INDIVIDUAL PROGNOSIS

KW - CARDIOVASCULAR RISK

U2 - 10.7326/M18-1376

DO - 10.7326/M18-1376

M3 - Article

SN - 0003-4819

VL - 170

SP - 51

EP - 60

JO - Annals of Internal Medicine

JF - Annals of Internal Medicine

IS - 1

ER -