The Enemies of Reliable and Useful Clinical Prediction Models: A Review of Statistical and Scientific Challenges

The current status of applied clinical prediction modeling is poor. Many models are developed with suboptimal methods and are not evaluated, and hence have little impact on clinical care. We review 12 challenges-provocatively labeled enemies-that jeopardize the creation of prediction models that make it to clinical practice to improve treatment decisions and clinical outcomes for individual patients. The challenges cover four areas: context, data, design and analysis, and scientific culture. We provide negative examples and recommendations for improvement, but also highlight positive examples and developments. Greater awareness of the complexities surrounding clinical prediction modeling is needed among researchers, funding agencies, health professionals as end users, and all of us as potential patients. To improve the utility of prediction models for healthcare and society, we need fewer but better models as well as more resources for model validation, impact assessment, and implementation.