Test-Retest Data for Radiomics Feature Stability Analysis: Generalizable or Study-Specific?

Janna E. van Timmeren*, Ralph T. H. Leijenaar, Wouter van Elmpt, Jiazhou Wang, Zhen Zhang, Andre Dekker, Philippe Lambin

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review


Radiomics is an objective method for extracting quantitative information from medical images. However, in radiomics, standardization, overfitting, and generalization are major challenges to be overcome. Test-retest experiments can be used to select robust radiomic features that have minimal variation. Currently, it is unknown whether they should be identified for each disease (disease specific) or are only imaging device-specific (computed tomography [CT]-specific). Here, we performed a test-retest analysis on CT scans of 40 patients with rectal cancer in a clinical setting. Correlation between radiomic features was assessed using the concordance correlation coefficient (CCC). In total, only 9/542 features have a CCC > 0.85. Furthermore, results were compared with the test-retest results on CT scans of 27 patients with lung cancer with a 15-minute interval. Results show that 446/542 features have a higher CCC for the test-retest analysis of the data set of patients with lung cancer than for patients with rectal cancer. The importance of controlling factors such as scanners, imaging protocol, reconstruction methods, and time points in a radiomics analysis is shown. Moreover, the results imply that test-retest analyses should be performed before each radiomics study. More research is required to independently evaluate the effect of each factor.
Original languageEnglish
Pages (from-to)361-365
Issue number4
Publication statusPublished - Dec 2016


  • radiomics
  • test-retest
  • computed tomography

Cite this