Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility

N.W. Schurink; S.R. van Kranen; S. Roberti; J.J.M. van Griethuysen; N. Bogveradze; F. Castagnoli; N. El Khababi; F.C.H. Bakers; S.H. de Bie; G.P.T. Bosma; V.C. Cappendijk; R.W.F. Geenen; P.A. Neijenhuis; G.M. Peterson; C.J. Veeken; R.F.A. Vliegen; R.G.H. Beets-Tan; D.M.J. Lambregts

doi:10.1007/s00330-021-08251-8

Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility

N.W. Schurink, S.R. van Kranen, S. Roberti, J.J.M. van Griethuysen, N. Bogveradze, F. Castagnoli, N. El Khababi, F.C.H. Bakers, S.H. de Bie, G.P.T. Bosma, V.C. Cappendijk, R.W.F. Geenen, P.A. Neijenhuis, G.M. Peterson, C.J. Veeken, R.F.A. Vliegen, R.G.H. Beets-Tan^*, D.M.J. Lambregts^*

^*Corresponding author for this work

Research output: Contribution to journal › Article › Academic › peer-review

Abstract

Objectives To investigate sources of variation in a multicenter rectal cancer MRI dataset focusing on hardware and image acquisition, segmentation methodology, and radiomics feature extraction software. Methods T2W and DWI/ADC MRIs from 649 rectal cancer patients were retrospectively acquired in 9 centers. Fifty-two imaging features (14 first-order/6 shape/32 higher-order) were extracted from each scan using whole-volume (expert/non-expert) and single-slice segmentations using two different software packages (PyRadiomics/CapTk). Influence of hardware, acquisition, and patient-intrinsic factors (age/gender/cTN-stage) on ADC was assessed using linear regression. Feature reproducibility was assessed between segmentation methods and software packages using the intraclass correlation coefficient. Results Image features differed significantly (p < 0.001) between centers with more substantial variations in ADC compared to T2W-MRI. In total, 64.3% of the variation in mean ADC was explained by differences in hardware and acquisition, compared to 0.4% by patient-intrinsic factors. Feature reproducibility between expert and non-expert segmentations was good to excellent (median ICC 0.89-0.90). Reproducibility for single-slice versus whole-volume segmentations was substantially poorer (median ICC 0.40-0.58). Between software packages, reproducibility was good to excellent (median ICC 0.99) for most features (first-order/shape/GLCM/GLRLM) but poor for higher-order (GLSZM/NGTDM) features (median ICC 0.00-0.41). Conclusions Significant variations are present in multicenter MRI data, particularly related to differences in hardware and acquisition, which will likely negatively influence subsequent analysis if not corrected for. Segmentation variations had a minor impact when using whole volume segmentations. Between software packages, higher-order features were less reproducible and caution is warranted when implementing these in prediction models.

Original language	English
Pages (from-to)	1506-1516
Number of pages	11
Journal	European Radiology
Volume	32
Issue number	3
Early online date	16 Oct 2021
DOIs	https://doi.org/10.1007/s00330-021-08251-8
Publication status	Published - Mar 2022

Keywords

Multicenter study
Rectal neoplasms
Reproducibility of results
Magnetic resonance imaging
Image processing
Computer-assisted
CANCER
PREDICTION
REPEATABILITY

Access to Document

10.1007/s00330-021-08251-8Licence: CC BY

Cite this

Schurink, N. W., van Kranen, S. R., Roberti, S., van Griethuysen, J. J. M., Bogveradze, N., Castagnoli, F., El Khababi, N., Bakers, F. C. H., de Bie, S. H., Bosma, G. P. T., Cappendijk, V. C., Geenen, R. W. F., Neijenhuis, P. A., Peterson, G. M., Veeken, C. J., Vliegen, R. F. A., Beets-Tan, R. G. H., & Lambregts, D. M. J. (2022). Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility. European Radiology, 32(3), 1506-1516. https://doi.org/10.1007/s00330-021-08251-8

@article{6713df53014047159b7fc8fed3180039,

title = "Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility",

abstract = "Objectives To investigate sources of variation in a multicenter rectal cancer MRI dataset focusing on hardware and image acquisition, segmentation methodology, and radiomics feature extraction software. Methods T2W and DWI/ADC MRIs from 649 rectal cancer patients were retrospectively acquired in 9 centers. Fifty-two imaging features (14 first-order/6 shape/32 higher-order) were extracted from each scan using whole-volume (expert/non-expert) and single-slice segmentations using two different software packages (PyRadiomics/CapTk). Influence of hardware, acquisition, and patient-intrinsic factors (age/gender/cTN-stage) on ADC was assessed using linear regression. Feature reproducibility was assessed between segmentation methods and software packages using the intraclass correlation coefficient. Results Image features differed significantly (p < 0.001) between centers with more substantial variations in ADC compared to T2W-MRI. In total, 64.3% of the variation in mean ADC was explained by differences in hardware and acquisition, compared to 0.4% by patient-intrinsic factors. Feature reproducibility between expert and non-expert segmentations was good to excellent (median ICC 0.89-0.90). Reproducibility for single-slice versus whole-volume segmentations was substantially poorer (median ICC 0.40-0.58). Between software packages, reproducibility was good to excellent (median ICC 0.99) for most features (first-order/shape/GLCM/GLRLM) but poor for higher-order (GLSZM/NGTDM) features (median ICC 0.00-0.41). Conclusions Significant variations are present in multicenter MRI data, particularly related to differences in hardware and acquisition, which will likely negatively influence subsequent analysis if not corrected for. Segmentation variations had a minor impact when using whole volume segmentations. Between software packages, higher-order features were less reproducible and caution is warranted when implementing these in prediction models.",

keywords = "Multicenter study, Rectal neoplasms, Reproducibility of results, Magnetic resonance imaging, Image processing, Computer-assisted, CANCER, PREDICTION, REPEATABILITY",

author = "N.W. Schurink and {van Kranen}, S.R. and S. Roberti and {van Griethuysen}, J.J.M. and N. Bogveradze and F. Castagnoli and {El Khababi}, N. and F.C.H. Bakers and {de Bie}, S.H. and G.P.T. Bosma and V.C. Cappendijk and R.W.F. Geenen and P.A. Neijenhuis and G.M. Peterson and C.J. Veeken and R.F.A. Vliegen and R.G.H. Beets-Tan and D.M.J. Lambregts",

note = "Funding Information: This study has received funding from the Dutch Cancer Society (project number 10138). Publisher Copyright: {\textcopyright} 2021, The Author(s).",

year = "2022",

month = mar,

doi = "10.1007/s00330-021-08251-8",

language = "English",

volume = "32",

pages = "1506--1516",

journal = "European Radiology",

issn = "0938-7994",

publisher = "Springer, Cham",

number = "3",

}

Schurink, NW, van Kranen, SR, Roberti, S, van Griethuysen, JJM, Bogveradze, N, Castagnoli, F, El Khababi, N , Bakers, FCH, de Bie, SH, Bosma, GPT, Cappendijk, VC, Geenen, RWF, Neijenhuis, PA, Peterson, GM, Veeken, CJ, Vliegen, RFA, Beets-Tan, RGH & Lambregts, DMJ 2022, 'Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility', European Radiology, vol. 32, no. 3, pp. 1506-1516. https://doi.org/10.1007/s00330-021-08251-8

TY - JOUR

T1 - Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility

AU - Schurink, N.W.

AU - van Kranen, S.R.

AU - Roberti, S.

AU - van Griethuysen, J.J.M.

AU - Bogveradze, N.

AU - Castagnoli, F.

AU - El Khababi, N.

AU - Bakers, F.C.H.

AU - de Bie, S.H.

AU - Bosma, G.P.T.

AU - Cappendijk, V.C.

AU - Geenen, R.W.F.

AU - Neijenhuis, P.A.

AU - Peterson, G.M.

AU - Veeken, C.J.

AU - Vliegen, R.F.A.

AU - Beets-Tan, R.G.H.

AU - Lambregts, D.M.J.

PY - 2022/3

Y1 - 2022/3

N2 - Objectives To investigate sources of variation in a multicenter rectal cancer MRI dataset focusing on hardware and image acquisition, segmentation methodology, and radiomics feature extraction software. Methods T2W and DWI/ADC MRIs from 649 rectal cancer patients were retrospectively acquired in 9 centers. Fifty-two imaging features (14 first-order/6 shape/32 higher-order) were extracted from each scan using whole-volume (expert/non-expert) and single-slice segmentations using two different software packages (PyRadiomics/CapTk). Influence of hardware, acquisition, and patient-intrinsic factors (age/gender/cTN-stage) on ADC was assessed using linear regression. Feature reproducibility was assessed between segmentation methods and software packages using the intraclass correlation coefficient. Results Image features differed significantly (p < 0.001) between centers with more substantial variations in ADC compared to T2W-MRI. In total, 64.3% of the variation in mean ADC was explained by differences in hardware and acquisition, compared to 0.4% by patient-intrinsic factors. Feature reproducibility between expert and non-expert segmentations was good to excellent (median ICC 0.89-0.90). Reproducibility for single-slice versus whole-volume segmentations was substantially poorer (median ICC 0.40-0.58). Between software packages, reproducibility was good to excellent (median ICC 0.99) for most features (first-order/shape/GLCM/GLRLM) but poor for higher-order (GLSZM/NGTDM) features (median ICC 0.00-0.41). Conclusions Significant variations are present in multicenter MRI data, particularly related to differences in hardware and acquisition, which will likely negatively influence subsequent analysis if not corrected for. Segmentation variations had a minor impact when using whole volume segmentations. Between software packages, higher-order features were less reproducible and caution is warranted when implementing these in prediction models.

AB - Objectives To investigate sources of variation in a multicenter rectal cancer MRI dataset focusing on hardware and image acquisition, segmentation methodology, and radiomics feature extraction software. Methods T2W and DWI/ADC MRIs from 649 rectal cancer patients were retrospectively acquired in 9 centers. Fifty-two imaging features (14 first-order/6 shape/32 higher-order) were extracted from each scan using whole-volume (expert/non-expert) and single-slice segmentations using two different software packages (PyRadiomics/CapTk). Influence of hardware, acquisition, and patient-intrinsic factors (age/gender/cTN-stage) on ADC was assessed using linear regression. Feature reproducibility was assessed between segmentation methods and software packages using the intraclass correlation coefficient. Results Image features differed significantly (p < 0.001) between centers with more substantial variations in ADC compared to T2W-MRI. In total, 64.3% of the variation in mean ADC was explained by differences in hardware and acquisition, compared to 0.4% by patient-intrinsic factors. Feature reproducibility between expert and non-expert segmentations was good to excellent (median ICC 0.89-0.90). Reproducibility for single-slice versus whole-volume segmentations was substantially poorer (median ICC 0.40-0.58). Between software packages, reproducibility was good to excellent (median ICC 0.99) for most features (first-order/shape/GLCM/GLRLM) but poor for higher-order (GLSZM/NGTDM) features (median ICC 0.00-0.41). Conclusions Significant variations are present in multicenter MRI data, particularly related to differences in hardware and acquisition, which will likely negatively influence subsequent analysis if not corrected for. Segmentation variations had a minor impact when using whole volume segmentations. Between software packages, higher-order features were less reproducible and caution is warranted when implementing these in prediction models.

KW - Multicenter study

KW - Rectal neoplasms

KW - Reproducibility of results

KW - Magnetic resonance imaging

KW - Image processing

KW - Computer-assisted

KW - CANCER

KW - PREDICTION

KW - REPEATABILITY

U2 - 10.1007/s00330-021-08251-8

DO - 10.1007/s00330-021-08251-8

M3 - Article

C2 - 34655313

SN - 0938-7994

VL - 32

SP - 1506

EP - 1516

JO - European Radiology

JF - European Radiology

IS - 3

ER -