The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping

Alex Zwanenburg; Martin Vallieres; Mahmoud A. Abdalah; Hugo J. W. L. Aerts; Vincent Andrearczyk; Aditya Apte; Saeed Ashrafinia; Spyridon Bakas; Roeloff Beukinga; Ronald Boellaard; Marta Bogowicz; Luca Boldrini; Irene Buvat; Gary J. R. Cook; Christos Davatzikos; Adrien Depeursinge; Marie-Charlotte Desseroit; Nicola Dinapoli; null Cuong Viet Dinh; Sebastian Echegaray; Philippe Lambin; Ralph T. H. Leijenaar; Joost van Griethuysen

doi:10.1148/radiol.2020191145

The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping

Alex Zwanenburg^*, Martin Vallieres, Mahmoud A. Abdalah, Hugo J. W. L. Aerts, Vincent Andrearczyk, Aditya Apte, Saeed Ashrafinia, Spyridon Bakas, Roeloff Beukinga, Ronald Boellaard, Marta Bogowicz, Luca Boldrini, Irene Buvat, Gary J. R. Cook, Christos Davatzikos, Adrien Depeursinge, Marie-Charlotte Desseroit, Nicola Dinapoli, Cuong Viet Dinh, Sebastian EchegarayPhilippe Lambin, Ralph T. H. Leijenaar, Joost van Griethuysen

^*Corresponding author for this work

Research output: Contribution to journal › Article › Academic › peer-review

Abstract

Background: Radiomic features may quantify characteristics present in medical imaging. However, the lack of standardized definitions and validated reference values have hampered clinical use.

Purpose: To standardize a set of 174 radiomic features.

Materials and Methods: Radiomic features were assessed in three phases. In phase I, 487 features were derived from the basic set of 174 features. Twenty-five research teams with unique radiomics software implementations computed feature values directly from a digital phantom, without any additional image processing. In phase II, 15 teams computed values for 1347 derived features using a CT image of a patient with lung cancer and predefined image processing configurations. In both phases, consensus among the teams on the validity of tentative reference values was measured through the frequency of the modal value and classified as follows: less than three matches, weak; three to five matches, moderate; six to nine matches, strong; 10 or more matches, very strong. In the final phase (phase III), a public data set of multimodality images (CT, fluorine 18 fluorodeoxyglucose PET, and T1-weighted MRI) from 51 patients with soft-tissue sarcoma was used to prospectively assess reproducibility of standardized features.

Results: Consensus on reference values was initially weak for 232 of 302 features (76.8%) at phase I and 703 of 1075 features (65.4%) at phase II. At the final iteration, weak consensus remained for only two of 487 features (0.4%) at phase I and 19 of 1347 features (1.4%) at phase II. Strong or better consensus was achieved for 463 of 487 features (95.1%) at phase I and 1220 of 1347 features (90.6%) at phase II. Overall, 169 of 174 features were standardized in the first two phases. In the final validation phase (phase III), most of the 169 standardized features could be excellently reproduced (166 with CT; 164 with PET; and 164 with MRI).

Conclusion: A set of 169 radiomics features was standardized, which enabled verification and calibration of different radiomics software. (C) RSNA, 2020

Original language	English
Pages (from-to)	328-338
Number of pages	11
Journal	Radiology
Volume	295
Issue number	2
DOIs	https://doi.org/10.1148/radiol.2020191145
Publication status	Published - May 2020

Keywords

CANCER
MODEL
PET
PREDICTION
TEXTURE ANALYSIS

Access to Document

10.1148/radiol.2020191145

Cite this

Zwanenburg, A., Vallieres, M., Abdalah, M. A., Aerts, H. J. W. L., Andrearczyk, V., Apte, A., Ashrafinia, S., Bakas, S., Beukinga, R., Boellaard, R., Bogowicz, M., Boldrini, L., Buvat, I., Cook, G. J. R., Davatzikos, C., Depeursinge, A., Desseroit, M.-C., Dinapoli, N., Cuong Viet Dinh, ... van Griethuysen, J. (2020). The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping. Radiology, 295(2), 328-338. https://doi.org/10.1148/radiol.2020191145

@article{a842e562d0514db29b40ef1d54c71af4,

title = "The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping",

abstract = "Background: Radiomic features may quantify characteristics present in medical imaging. However, the lack of standardized definitions and validated reference values have hampered clinical use.Purpose: To standardize a set of 174 radiomic features.Materials and Methods: Radiomic features were assessed in three phases. In phase I, 487 features were derived from the basic set of 174 features. Twenty-five research teams with unique radiomics software implementations computed feature values directly from a digital phantom, without any additional image processing. In phase II, 15 teams computed values for 1347 derived features using a CT image of a patient with lung cancer and predefined image processing configurations. In both phases, consensus among the teams on the validity of tentative reference values was measured through the frequency of the modal value and classified as follows: less than three matches, weak; three to five matches, moderate; six to nine matches, strong; 10 or more matches, very strong. In the final phase (phase III), a public data set of multimodality images (CT, fluorine 18 fluorodeoxyglucose PET, and T1-weighted MRI) from 51 patients with soft-tissue sarcoma was used to prospectively assess reproducibility of standardized features.Results: Consensus on reference values was initially weak for 232 of 302 features (76.8%) at phase I and 703 of 1075 features (65.4%) at phase II. At the final iteration, weak consensus remained for only two of 487 features (0.4%) at phase I and 19 of 1347 features (1.4%) at phase II. Strong or better consensus was achieved for 463 of 487 features (95.1%) at phase I and 1220 of 1347 features (90.6%) at phase II. Overall, 169 of 174 features were standardized in the first two phases. In the final validation phase (phase III), most of the 169 standardized features could be excellently reproduced (166 with CT; 164 with PET; and 164 with MRI).Conclusion: A set of 169 radiomics features was standardized, which enabled verification and calibration of different radiomics software. (C) RSNA, 2020",

keywords = "CANCER, MODEL, PET, PREDICTION, TEXTURE ANALYSIS",

author = "Alex Zwanenburg and Martin Vallieres and Abdalah, {Mahmoud A.} and Aerts, {Hugo J. W. L.} and Vincent Andrearczyk and Aditya Apte and Saeed Ashrafinia and Spyridon Bakas and Roeloff Beukinga and Ronald Boellaard and Marta Bogowicz and Luca Boldrini and Irene Buvat and Cook, {Gary J. R.} and Christos Davatzikos and Adrien Depeursinge and Marie-Charlotte Desseroit and Nicola Dinapoli and {Cuong Viet Dinh} and Sebastian Echegaray and Philippe Lambin and Leijenaar, {Ralph T. H.} and {van Griethuysen}, Joost",

note = "Funding Information: Author contributions: Guarantors of integrity of entire study, A.Z., M.A.A., A.D., I.E.N., P.L., O.M., S.N., S.P., E.A.G.P., J.S.F., E.S., R.J.H.M.S., T.U., L.V.v.D.; study concepts/study design or data acquisition or data analysis/interpretation, all authors; manuscript drafting or manuscript revision for important intellectual content, all authors; approval of final version of submitted manuscript, all authors; agrees to ensure any questions related to the work are appropriately resolved, all authors; literature research, A.Z., M.V., M.A.A., H.J.W.L.A., G.J.R.C., C.D., N.D., C.V.D., I.E.N., A.Y.F., M.H., O.M., H.M., S.P., A.U.K.R., M.M.S., J.S.F., R.J.H.M.S., D.T., E.G.C.T., V.V., F.H.P.v.V., C.R.; clinical studies, G.J.R.C., N.D., I.E.N., A.Y.F., S.P., J.S.F., R.J.H.M.S., V.V., F.H.P.v.V.; experimental studies, A.Z., M.V., M.A.A., H.J.W.L.A., V.A., A.A., S.A., S.B., R.J.B., R.B., M.B., L.B., G.J.R.C., C.D., A.D., M.C.D., N.D., C.V.D., S.E., R.G., R.J.G., M. Guckenberg-er, M. G{\"o}tz, S.M.H., P.L., S. Leger, R.T.H.L., K.H.M.H., O.M., H.M., S.N., C.N., F.O., S.P., A.R., J.S., M.M.S., J.S.F., E.S., R.J.H.M.S., D.T., T.U., V.V., L.V.v.D., J.v.G., F.H.P.v.V.; statistical analysis, A.Z., M.A.A., S.B., I.B., N.D., R.J.G., S.M.H., O.M., H.M., S.P., E.A.G.P., M.M.S., J.S.F., R.J.H.M.S., D.T., V.V.; and manuscript editing, A.Z., M.V., M.A.A., H.J.W.L.A., V.A., A.A., S.B., R.J.B., R.B., M.B., L.B., I.B., G.J.R.C., C.D., A.D., N.D., I.E.N., A.Y.F., R.J.G., V.G., M. Gu-ckenberger, M. G{\"o}tz, M.H., P.L., S. Leger, R.T.H.L., K.H.M.H., O.M., H.M., S.N., S.P., E.A.G.P., A.U.K.R., M.M.S., N.M.S., J.S.F., E.S., R.J.H.M.S., D.T., E.G.C.T., V.V., L.V.v.D., J.v.G., F.H.P.v.V., C.R., S. L{\"o}ck The authors received funding from the Cancer Research UK and Engineering and Physical Sciences Research Council, with the Medical Research Council and the Department of Health and Social Care (C1519/A16463: M.M.S., G.C., V.G.), Dutch Cancer Society (10034: R.B.), EU Seventh Framework Programme (ARTFORCE 257144: R.T.H.L., P.L.; REQUITE 601826: R.T.H.L., P.L.), Engineering and Physical Sciences Research Council (EP/M507842/1: P.W., E.S.; EP/N509449/1: P.W., E.S.), European Research Council (ERC AdG-2015: 694812-Hypoximmuno: R.T.H.L., P.L.; ERC StG-2013: 335367 bio-iRT: D.T.), Eurostars (DART 10116: R.T.H.L., P.L.; DECIDE 11541: R.T.H.L., P.L.), French National Institute of Cancer (C14020NS: M.C.D., M.H.), French National Research Agency (ANR-10-LABX-07-01: M.C.D., M.H.; ANR-11-IDEX-0003-02: C.N., F.O., I.B.), German Federal Ministry of Education and Research (BMBF-03Z1N52: A.Z., S. Leger, E.G.C.T, C.R.), Horizon 2020 Framework Programmme (BD2Decide PHC-30-689715: R.T.H.L., P.L.; IMMUNOSABR SC1-PM-733008: R.T.H.L., P.L.), Innovative Medicines Initiative (IMI JU QuIC-ConCePT 115151: R.T.H.L., P.L.), Interreg V-A Euregio Meuse-Rhine (Euradiomics: R.T.H.L., P.L.), National Cancer Institute (P30CA008748: A.A.; U01CA187947: S.E., S.N.; U24CA189523: S.B., S.P., S.M.H., C.D.), National Institute of Neurologic Disorders and Stroke (R01NS042645: S.B., S.P., S.M.H., C.D.), National Institutes of Health (R01CA198121: A.A.; U01CA143062: R.J.G.; U01CA190234: J.v.G., A.Y.F., H.J.W.L.A.; U24CA180918: A.Y.F.; U24CA194354: J.v.G., A.Y.F., H.J.W.L.A.), SME phase 2 (RAIL 673780: R.T.H.L., P.L.), Swiss National Science Foundation (310030 173303: M.B., S.T.L., M. Guckenberger; PZ00P2 154891: A.D.), Technology Foundation STW (10696 DuCAT: R.T.H.L., P.L.; P14-19 Radiomics STRaTegy: R.T.H.L., P.L.), the Netherlands Organization for Health Research and Development (10-10400-98-14002: R.B.), the Netherlands Organization for Scientific Research (14929: E.A.G.P., R.B.), University of Zurich Clinical Research Priority Program (Tumor Oxygenation: M.B., S.T.L., M. Guckenberger), and the Wellcome Trust (WT203148/Z/16/Z: M.M.S., G.C., V.G.). Publisher Copyright: {\textcopyright} RSNA, 2020.",

year = "2020",

month = may,

doi = "10.1148/radiol.2020191145",

language = "English",

volume = "295",

pages = "328--338",

journal = "Radiology",

issn = "0033-8419",

publisher = "Radiological Society of North America, Inc.",

number = "2",

}

Zwanenburg, A, Vallieres, M, Abdalah, MA, Aerts, HJWL, Andrearczyk, V, Apte, A, Ashrafinia, S, Bakas, S, Beukinga, R, Boellaard, R, Bogowicz, M, Boldrini, L, Buvat, I, Cook, GJR, Davatzikos, C, Depeursinge, A, Desseroit, M-C, Dinapoli, N, Cuong Viet Dinh, Echegaray, S, Lambin, P , Leijenaar, RTH & van Griethuysen, J 2020, 'The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping', Radiology, vol. 295, no. 2, pp. 328-338. https://doi.org/10.1148/radiol.2020191145

TY - JOUR

T1 - The Image Biomarker Standardization Initiative

T2 - Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping

AU - Zwanenburg, Alex

AU - Vallieres, Martin

AU - Abdalah, Mahmoud A.

AU - Aerts, Hugo J. W. L.

AU - Andrearczyk, Vincent

AU - Apte, Aditya

AU - Ashrafinia, Saeed

AU - Bakas, Spyridon

AU - Beukinga, Roeloff

AU - Boellaard, Ronald

AU - Bogowicz, Marta

AU - Boldrini, Luca

AU - Buvat, Irene

AU - Cook, Gary J. R.

AU - Davatzikos, Christos

AU - Depeursinge, Adrien

AU - Desseroit, Marie-Charlotte

AU - Dinapoli, Nicola

AU - Cuong Viet Dinh, null

AU - Echegaray, Sebastian

AU - Lambin, Philippe

AU - Leijenaar, Ralph T. H.

AU - van Griethuysen, Joost

N1 - Funding Information: Author contributions: Guarantors of integrity of entire study, A.Z., M.A.A., A.D., I.E.N., P.L., O.M., S.N., S.P., E.A.G.P., J.S.F., E.S., R.J.H.M.S., T.U., L.V.v.D.; study concepts/study design or data acquisition or data analysis/interpretation, all authors; manuscript drafting or manuscript revision for important intellectual content, all authors; approval of final version of submitted manuscript, all authors; agrees to ensure any questions related to the work are appropriately resolved, all authors; literature research, A.Z., M.V., M.A.A., H.J.W.L.A., G.J.R.C., C.D., N.D., C.V.D., I.E.N., A.Y.F., M.H., O.M., H.M., S.P., A.U.K.R., M.M.S., J.S.F., R.J.H.M.S., D.T., E.G.C.T., V.V., F.H.P.v.V., C.R.; clinical studies, G.J.R.C., N.D., I.E.N., A.Y.F., S.P., J.S.F., R.J.H.M.S., V.V., F.H.P.v.V.; experimental studies, A.Z., M.V., M.A.A., H.J.W.L.A., V.A., A.A., S.A., S.B., R.J.B., R.B., M.B., L.B., G.J.R.C., C.D., A.D., M.C.D., N.D., C.V.D., S.E., R.G., R.J.G., M. Guckenberg-er, M. Götz, S.M.H., P.L., S. Leger, R.T.H.L., K.H.M.H., O.M., H.M., S.N., C.N., F.O., S.P., A.R., J.S., M.M.S., J.S.F., E.S., R.J.H.M.S., D.T., T.U., V.V., L.V.v.D., J.v.G., F.H.P.v.V.; statistical analysis, A.Z., M.A.A., S.B., I.B., N.D., R.J.G., S.M.H., O.M., H.M., S.P., E.A.G.P., M.M.S., J.S.F., R.J.H.M.S., D.T., V.V.; and manuscript editing, A.Z., M.V., M.A.A., H.J.W.L.A., V.A., A.A., S.B., R.J.B., R.B., M.B., L.B., I.B., G.J.R.C., C.D., A.D., N.D., I.E.N., A.Y.F., R.J.G., V.G., M. Gu-ckenberger, M. Götz, M.H., P.L., S. Leger, R.T.H.L., K.H.M.H., O.M., H.M., S.N., S.P., E.A.G.P., A.U.K.R., M.M.S., N.M.S., J.S.F., E.S., R.J.H.M.S., D.T., E.G.C.T., V.V., L.V.v.D., J.v.G., F.H.P.v.V., C.R., S. Löck The authors received funding from the Cancer Research UK and Engineering and Physical Sciences Research Council, with the Medical Research Council and the Department of Health and Social Care (C1519/A16463: M.M.S., G.C., V.G.), Dutch Cancer Society (10034: R.B.), EU Seventh Framework Programme (ARTFORCE 257144: R.T.H.L., P.L.; REQUITE 601826: R.T.H.L., P.L.), Engineering and Physical Sciences Research Council (EP/M507842/1: P.W., E.S.; EP/N509449/1: P.W., E.S.), European Research Council (ERC AdG-2015: 694812-Hypoximmuno: R.T.H.L., P.L.; ERC StG-2013: 335367 bio-iRT: D.T.), Eurostars (DART 10116: R.T.H.L., P.L.; DECIDE 11541: R.T.H.L., P.L.), French National Institute of Cancer (C14020NS: M.C.D., M.H.), French National Research Agency (ANR-10-LABX-07-01: M.C.D., M.H.; ANR-11-IDEX-0003-02: C.N., F.O., I.B.), German Federal Ministry of Education and Research (BMBF-03Z1N52: A.Z., S. Leger, E.G.C.T, C.R.), Horizon 2020 Framework Programmme (BD2Decide PHC-30-689715: R.T.H.L., P.L.; IMMUNOSABR SC1-PM-733008: R.T.H.L., P.L.), Innovative Medicines Initiative (IMI JU QuIC-ConCePT 115151: R.T.H.L., P.L.), Interreg V-A Euregio Meuse-Rhine (Euradiomics: R.T.H.L., P.L.), National Cancer Institute (P30CA008748: A.A.; U01CA187947: S.E., S.N.; U24CA189523: S.B., S.P., S.M.H., C.D.), National Institute of Neurologic Disorders and Stroke (R01NS042645: S.B., S.P., S.M.H., C.D.), National Institutes of Health (R01CA198121: A.A.; U01CA143062: R.J.G.; U01CA190234: J.v.G., A.Y.F., H.J.W.L.A.; U24CA180918: A.Y.F.; U24CA194354: J.v.G., A.Y.F., H.J.W.L.A.), SME phase 2 (RAIL 673780: R.T.H.L., P.L.), Swiss National Science Foundation (310030 173303: M.B., S.T.L., M. Guckenberger; PZ00P2 154891: A.D.), Technology Foundation STW (10696 DuCAT: R.T.H.L., P.L.; P14-19 Radiomics STRaTegy: R.T.H.L., P.L.), the Netherlands Organization for Health Research and Development (10-10400-98-14002: R.B.), the Netherlands Organization for Scientific Research (14929: E.A.G.P., R.B.), University of Zurich Clinical Research Priority Program (Tumor Oxygenation: M.B., S.T.L., M. Guckenberger), and the Wellcome Trust (WT203148/Z/16/Z: M.M.S., G.C., V.G.). Publisher Copyright: © RSNA, 2020.

PY - 2020/5

Y1 - 2020/5

N2 - Background: Radiomic features may quantify characteristics present in medical imaging. However, the lack of standardized definitions and validated reference values have hampered clinical use.Purpose: To standardize a set of 174 radiomic features.Materials and Methods: Radiomic features were assessed in three phases. In phase I, 487 features were derived from the basic set of 174 features. Twenty-five research teams with unique radiomics software implementations computed feature values directly from a digital phantom, without any additional image processing. In phase II, 15 teams computed values for 1347 derived features using a CT image of a patient with lung cancer and predefined image processing configurations. In both phases, consensus among the teams on the validity of tentative reference values was measured through the frequency of the modal value and classified as follows: less than three matches, weak; three to five matches, moderate; six to nine matches, strong; 10 or more matches, very strong. In the final phase (phase III), a public data set of multimodality images (CT, fluorine 18 fluorodeoxyglucose PET, and T1-weighted MRI) from 51 patients with soft-tissue sarcoma was used to prospectively assess reproducibility of standardized features.Results: Consensus on reference values was initially weak for 232 of 302 features (76.8%) at phase I and 703 of 1075 features (65.4%) at phase II. At the final iteration, weak consensus remained for only two of 487 features (0.4%) at phase I and 19 of 1347 features (1.4%) at phase II. Strong or better consensus was achieved for 463 of 487 features (95.1%) at phase I and 1220 of 1347 features (90.6%) at phase II. Overall, 169 of 174 features were standardized in the first two phases. In the final validation phase (phase III), most of the 169 standardized features could be excellently reproduced (166 with CT; 164 with PET; and 164 with MRI).Conclusion: A set of 169 radiomics features was standardized, which enabled verification and calibration of different radiomics software. (C) RSNA, 2020

AB - Background: Radiomic features may quantify characteristics present in medical imaging. However, the lack of standardized definitions and validated reference values have hampered clinical use.Purpose: To standardize a set of 174 radiomic features.Materials and Methods: Radiomic features were assessed in three phases. In phase I, 487 features were derived from the basic set of 174 features. Twenty-five research teams with unique radiomics software implementations computed feature values directly from a digital phantom, without any additional image processing. In phase II, 15 teams computed values for 1347 derived features using a CT image of a patient with lung cancer and predefined image processing configurations. In both phases, consensus among the teams on the validity of tentative reference values was measured through the frequency of the modal value and classified as follows: less than three matches, weak; three to five matches, moderate; six to nine matches, strong; 10 or more matches, very strong. In the final phase (phase III), a public data set of multimodality images (CT, fluorine 18 fluorodeoxyglucose PET, and T1-weighted MRI) from 51 patients with soft-tissue sarcoma was used to prospectively assess reproducibility of standardized features.Results: Consensus on reference values was initially weak for 232 of 302 features (76.8%) at phase I and 703 of 1075 features (65.4%) at phase II. At the final iteration, weak consensus remained for only two of 487 features (0.4%) at phase I and 19 of 1347 features (1.4%) at phase II. Strong or better consensus was achieved for 463 of 487 features (95.1%) at phase I and 1220 of 1347 features (90.6%) at phase II. Overall, 169 of 174 features were standardized in the first two phases. In the final validation phase (phase III), most of the 169 standardized features could be excellently reproduced (166 with CT; 164 with PET; and 164 with MRI).Conclusion: A set of 169 radiomics features was standardized, which enabled verification and calibration of different radiomics software. (C) RSNA, 2020

KW - CANCER

KW - MODEL

KW - PET

KW - PREDICTION

KW - TEXTURE ANALYSIS

U2 - 10.1148/radiol.2020191145

DO - 10.1148/radiol.2020191145

M3 - Article

C2 - 32154773

SN - 0033-8419

VL - 295

SP - 328

EP - 338

JO - Radiology

JF - Radiology

IS - 2

ER -