The effect of imputing missing clinical attribute values on training lung cancer survival prediction model performance

Mohamed S. Barakat, Matthew Field*, Aditya Ghose, David Stirling, Lois Holloway, Shalini Vinod, Andre Dekker, David Thwaites

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

7 Citations (Web of Science)
6 Downloads (Pure)

Abstract

According to the estimations of the World Health Organization and the International Agency for Research in Cancer, lung cancer is the most common cause of death from cancer worldwide. The last few years have witnessed a rise in the attention given to the use of clinical decision support systems in medicine generally and in cancer in particular. These can predict patients' likelihood of survival based on analysis of and learning from previously treated patients. The datasets that are mined for developing clinical decision support functionality are often incomplete, which adversely impacts the quality of the models developed and the decision support offered. Imputing missing data using a statistical analysis approach is a common method to addressing the missing data problem. This work investigates the effect of imputation methods for missing data in preparing a training dataset for a Non-Small Cell Lung Cancer survival prediction model using several machine learning algorithms. The investigation includes an assessment of the effect of imputation algorithm error on performance prediction and also a comparison between using a smaller complete real dataset or a larger dataset with imputed data. Our results show that even when the proportion of records with some missing data is very high (> 80%) imputation can lead to prediction models with an AUC (0.68-0.72) comparable to those trained with complete data records.

Original languageEnglish
Article number16
Number of pages11
JournalHealth information science and systems
Volume5
DOIs
Publication statusPublished - 6 Dec 2017

Keywords

  • Missing data
  • Imputation
  • Decision Support
  • Modeling and Lung Cancer
  • IMPUTATION
  • CLASSIFICATION
  • SYSTEM
  • CARE

Cite this