Three myths about risk thresholds for prediction models

Laure Wynants*, Maarten van Smeden, David J. McLernon, Dirk Timmerman, Ewout W. Steyerberg, Ben Van Calster, Topic Group ‘Evaluating diagnostic tests and prediction models’ of the STRATOS initiative

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

Background Clinical prediction models are useful in estimating a patient's risk of having a certain disease or experiencing an event in the future based on their current characteristics. Defining an appropriate risk threshold to recommend intervention is a key challenge in bringing a risk prediction model to clinical application; such risk thresholds are often defined in an ad hoc way. This is problematic because tacitly assumed costs of false positive and false negative classifications may not be clinically sensible. For example, when choosing the risk threshold that maximizes the proportion of patients correctly classified, false positives and false negatives are assumed equally costly. Furthermore, small to moderate sample sizes may lead to unstable optimal thresholds, which requires a particularly cautious interpretation of results. Main text We discuss how three common myths about risk thresholds often lead to inappropriate risk stratification of patients. First, we point out the contexts of counseling and shared decision-making in which a continuous risk estimate is more useful than risk stratification. Second, we argue that threshold selection should reflect the consequences of the decisions made following risk stratification. Third, we emphasize that there is usually no universally optimal threshold but rather that a plausible risk threshold depends on the clinical context. Consequently, we recommend to present results for multiple risk thresholds when developing or validating a prediction model. Conclusion Bearing in mind these three considerations can avoid inappropriate allocation (and non-allocation) of interventions. Using discriminating and well-calibrated models will generate better clinical outcomes if context-dependent thresholds are used.

Original languageEnglish
Article number192
Number of pages7
JournalBMC Medicine
Volume17
Issue number1
DOIs
Publication statusPublished - 25 Oct 2019

Keywords

  • Clinical risk prediction model
  • Threshold
  • Decision support techniques
  • Risk
  • Data science
  • Diagnosis
  • Prognosis
  • ROC CURVE
  • CANCER
  • INDEX
  • SPECIFICITY
  • SENSITIVITY
  • PERFORMANCE
  • VALIDATION
  • MORTALITY
  • AREA

Fingerprint

Dive into the research topics of 'Three myths about risk thresholds for prediction models'. Together they form a unique fingerprint.

Cite this