Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy

Frank te Nijenhuis; Ruisheng Su; Pieter Jan van Doormaal; Jeannette Hofmeijer; Jasper Martens; Wim van Zwam; Aad van der Lugt; Theo van Walsum

doi:10.1007/978-3-031-76160-7_14

Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy

Frank te Nijenhuis^*, Ruisheng Su, Pieter Jan van Doormaal, Jeannette Hofmeijer, Jasper Martens, Wim van Zwam, Aad van der Lugt, Theo van Walsum

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceeding › Academic › peer-review

1 Downloads (Pure)

Abstract

The efficacy of endovascular therapy (EVT) in large vessel occlusion (LVO) of the anterior circulation depends on adequate patient selection. Patients can be selected based on their predicted functional outcome after EVT. Using a dataset composed of 1929 patients, we compare the functional outcome prediction performance of clinical baseline models, including the clinically validated MR PREDICTS decision tool, with an imaging based pipeline and a multimodal approach. The predicted outcome measure is dichotomized modified Rankin Scale score 90 days after mechanical thrombectomy. Binary classifier performance is quantified using Area-Under the receiver operating characteristic Curve (AUC). Combining clinical features with information extracted from CTA images does not significantly improve the performance of functional outcome prediction methods compared to the baseline model. This multimodal approach can however replace radiologically derived biomarkers, as its performance is non-inferior.

Original language	English
Title of host publication	Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 9th International Workshop, BrainLes 2023, and 3rd International Workshop, SWITCH 2023, Held in Conjunction with MICCAI 2023, Revised Selected Papers
Editors	Ujjwal Baid, Reuben Dorent, Sylwia Malec, Monika Pytlarz, Ruisheng Su, Navodini Wijethilake, Spyridon Bakas, Alessandro Crimi
Publisher	Springer Verlag
Pages	144-153
Number of pages	10
Volume	14668 LNCS
ISBN (Print)	9783031761591
DOIs	https://doi.org/10.1007/978-3-031-76160-7_14
Publication status	Published - 1 Jan 2024
Event	9th International Workshop on Brain Lesion workshop, BrainLes 2023 and 3rd Stroke Workshop on Imaging and Treatment CHallenges, SWITCH 2023 Held in Conjunction with 26th Medical Image Computing and Computer Assisted Intervention, MICCAI 2023 - Vancouver, Canada Duration: 8 Oct 2023 → 12 Oct 2023 https://switchmiccai.github.io/switch-2023/

Publication series

Series	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14668 LNCS
ISSN	0302-9743

Workshop

Workshop	9th International Workshop on Brain Lesion workshop, BrainLes 2023 and 3rd Stroke Workshop on Imaging and Treatment CHallenges, SWITCH 2023 Held in Conjunction with 26th Medical Image Computing and Computer Assisted Intervention, MICCAI 2023
Abbreviated title	SWITCH 2023
Country/Territory	Canada
City	Vancouver
Period	8/10/23 → 12/10/23
Internet address	https://switchmiccai.github.io/switch-2023/

Keywords

Artificial Intelligence
Deep Learning
Endovascular Therapy
Functional Outcome Prediction
Mechanical Thrombectomy
Med3D
MR CLEAN Registry
MR PREDICTS
Stroke

Access to Document

10.1007/978-3-031-76160-7_14

Embargoed Document

Full Text
Final published version, 1.06 MB
Licence: Taverne
Embargo ends: 27/06/25
Request copy

Cite this

te Nijenhuis, F., Su, R., van Doormaal, P. J., Hofmeijer, J., Martens, J., van Zwam, W., van der Lugt, A., & van Walsum, T. (2024). Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy. In U. Baid, R. Dorent, S. Malec, M. Pytlarz, R. Su, N. Wijethilake, S. Bakas, & A. Crimi (Eds.), Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 9th International Workshop, BrainLes 2023, and 3rd International Workshop, SWITCH 2023, Held in Conjunction with MICCAI 2023, Revised Selected Papers (Vol. 14668 LNCS, pp. 144-153). Springer Verlag. https://doi.org/10.1007/978-3-031-76160-7_14

te Nijenhuis, Frank ; Su, Ruisheng ; van Doormaal, Pieter Jan et al. / Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy. Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 9th International Workshop, BrainLes 2023, and 3rd International Workshop, SWITCH 2023, Held in Conjunction with MICCAI 2023, Revised Selected Papers. editor / Ujjwal Baid ; Reuben Dorent ; Sylwia Malec ; Monika Pytlarz ; Ruisheng Su ; Navodini Wijethilake ; Spyridon Bakas ; Alessandro Crimi. Vol. 14668 LNCS Springer Verlag, 2024. pp. 144-153 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 14668 LNCS).

@inproceedings{01ce5bd528694e3b8bdcf1b3477b3d81,

title = "Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy",

abstract = "The efficacy of endovascular therapy (EVT) in large vessel occlusion (LVO) of the anterior circulation depends on adequate patient selection. Patients can be selected based on their predicted functional outcome after EVT. Using a dataset composed of 1929 patients, we compare the functional outcome prediction performance of clinical baseline models, including the clinically validated MR PREDICTS decision tool, with an imaging based pipeline and a multimodal approach. The predicted outcome measure is dichotomized modified Rankin Scale score 90 days after mechanical thrombectomy. Binary classifier performance is quantified using Area-Under the receiver operating characteristic Curve (AUC). Combining clinical features with information extracted from CTA images does not significantly improve the performance of functional outcome prediction methods compared to the baseline model. This multimodal approach can however replace radiologically derived biomarkers, as its performance is non-inferior.",

keywords = "Artificial Intelligence, Deep Learning, Endovascular Therapy, Functional Outcome Prediction, Mechanical Thrombectomy, Med3D, MR CLEAN Registry, MR PREDICTS, Stroke",

author = "{te Nijenhuis}, Frank and Ruisheng Su and {van Doormaal}, {Pieter Jan} and Jeannette Hofmeijer and Jasper Martens and {van Zwam}, Wim and {van der Lugt}, Aad and {van Walsum}, Theo",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.; 9th International Workshop on Brain Lesion workshop, BrainLes 2023 and 3rd Stroke Workshop on Imaging and Treatment CHallenges, SWITCH 2023 Held in Conjunction with 26th Medical Image Computing and Computer Assisted Intervention, MICCAI 2023, SWITCH 2023 ; Conference date: 08-10-2023 Through 12-10-2023",

year = "2024",

month = jan,

day = "1",

doi = "10.1007/978-3-031-76160-7_14",

language = "English",

isbn = "9783031761591",

volume = "14668 LNCS",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "144--153",

editor = "Ujjwal Baid and Reuben Dorent and Sylwia Malec and Monika Pytlarz and Ruisheng Su and Navodini Wijethilake and Spyridon Bakas and Alessandro Crimi",

booktitle = "Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 9th International Workshop, BrainLes 2023, and 3rd International Workshop, SWITCH 2023, Held in Conjunction with MICCAI 2023, Revised Selected Papers",

address = "Germany",

url = "https://switchmiccai.github.io/switch-2023/",

}

te Nijenhuis, F, Su, R, van Doormaal, PJ, Hofmeijer, J, Martens, J, van Zwam, W, van der Lugt, A & van Walsum, T 2024, Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy. in U Baid, R Dorent, S Malec, M Pytlarz, R Su, N Wijethilake, S Bakas & A Crimi (eds), Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 9th International Workshop, BrainLes 2023, and 3rd International Workshop, SWITCH 2023, Held in Conjunction with MICCAI 2023, Revised Selected Papers. vol. 14668 LNCS, Springer Verlag, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14668 LNCS, pp. 144-153, 9th International Workshop on Brain Lesion workshop, BrainLes 2023 and 3rd Stroke Workshop on Imaging and Treatment CHallenges, SWITCH 2023 Held in Conjunction with 26th Medical Image Computing and Computer Assisted Intervention, MICCAI 2023, Vancouver, British Columbia, Canada, 8/10/23. https://doi.org/10.1007/978-3-031-76160-7_14

Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy. / te Nijenhuis, Frank; Su, Ruisheng; van Doormaal, Pieter Jan et al.
Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 9th International Workshop, BrainLes 2023, and 3rd International Workshop, SWITCH 2023, Held in Conjunction with MICCAI 2023, Revised Selected Papers. ed. / Ujjwal Baid; Reuben Dorent; Sylwia Malec; Monika Pytlarz; Ruisheng Su; Navodini Wijethilake; Spyridon Bakas; Alessandro Crimi. Vol. 14668 LNCS Springer Verlag, 2024. p. 144-153 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 14668 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceeding › Academic › peer-review

TY - GEN

T1 - Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy

AU - te Nijenhuis, Frank

AU - Su, Ruisheng

AU - van Doormaal, Pieter Jan

AU - Hofmeijer, Jeannette

AU - Martens, Jasper

AU - van Zwam, Wim

AU - van der Lugt, Aad

AU - van Walsum, Theo

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

PY - 2024/1/1

Y1 - 2024/1/1

N2 - The efficacy of endovascular therapy (EVT) in large vessel occlusion (LVO) of the anterior circulation depends on adequate patient selection. Patients can be selected based on their predicted functional outcome after EVT. Using a dataset composed of 1929 patients, we compare the functional outcome prediction performance of clinical baseline models, including the clinically validated MR PREDICTS decision tool, with an imaging based pipeline and a multimodal approach. The predicted outcome measure is dichotomized modified Rankin Scale score 90 days after mechanical thrombectomy. Binary classifier performance is quantified using Area-Under the receiver operating characteristic Curve (AUC). Combining clinical features with information extracted from CTA images does not significantly improve the performance of functional outcome prediction methods compared to the baseline model. This multimodal approach can however replace radiologically derived biomarkers, as its performance is non-inferior.

AB - The efficacy of endovascular therapy (EVT) in large vessel occlusion (LVO) of the anterior circulation depends on adequate patient selection. Patients can be selected based on their predicted functional outcome after EVT. Using a dataset composed of 1929 patients, we compare the functional outcome prediction performance of clinical baseline models, including the clinically validated MR PREDICTS decision tool, with an imaging based pipeline and a multimodal approach. The predicted outcome measure is dichotomized modified Rankin Scale score 90 days after mechanical thrombectomy. Binary classifier performance is quantified using Area-Under the receiver operating characteristic Curve (AUC). Combining clinical features with information extracted from CTA images does not significantly improve the performance of functional outcome prediction methods compared to the baseline model. This multimodal approach can however replace radiologically derived biomarkers, as its performance is non-inferior.

KW - Artificial Intelligence

KW - Deep Learning

KW - Endovascular Therapy

KW - Functional Outcome Prediction

KW - Mechanical Thrombectomy

KW - Med3D

KW - MR CLEAN Registry

KW - MR PREDICTS

KW - Stroke

U2 - 10.1007/978-3-031-76160-7_14

DO - 10.1007/978-3-031-76160-7_14

M3 - Conference article in proceeding

SN - 9783031761591

VL - 14668 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 144

EP - 153

BT - Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 9th International Workshop, BrainLes 2023, and 3rd International Workshop, SWITCH 2023, Held in Conjunction with MICCAI 2023, Revised Selected Papers

A2 - Baid, Ujjwal

A2 - Dorent, Reuben

A2 - Malec, Sylwia

A2 - Pytlarz, Monika

A2 - Su, Ruisheng

A2 - Wijethilake, Navodini

A2 - Bakas, Spyridon

A2 - Crimi, Alessandro

PB - Springer Verlag

T2 - 9th International Workshop on Brain Lesion workshop, BrainLes 2023 and 3rd Stroke Workshop on Imaging and Treatment CHallenges, SWITCH 2023 Held in Conjunction with 26th Medical Image Computing and Computer Assisted Intervention, MICCAI 2023

Y2 - 8 October 2023 through 12 October 2023

ER -

te Nijenhuis F, Su R, van Doormaal PJ, Hofmeijer J, Martens J, van Zwam W et al. Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy. In Baid U, Dorent R, Malec S, Pytlarz M, Su R, Wijethilake N, Bakas S, Crimi A, editors, Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 9th International Workshop, BrainLes 2023, and 3rd International Workshop, SWITCH 2023, Held in Conjunction with MICCAI 2023, Revised Selected Papers. Vol. 14668 LNCS. Springer Verlag. 2024. p. 144-153. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 14668 LNCS). doi: 10.1007/978-3-031-76160-7_14

Multimodal Deep Learning for Functional Outcome Prediction in Endovascular Therapy

Abstract

Publication series

Workshop

Keywords

Access to Document

Embargoed Document

Fingerprint

Cite this