Transformer-based biomarker prediction from colorectal cancer histology: A large-scale multicentric study

Sophia J Wagner; Daniel Reisenbüchler; Nicholas P West; Jan Moritz Niehues; Jiefu Zhu; Sebastian Foersch; Gregory Patrick Veldhuizen; Philip Quirke; Heike I Grabsch; Piet A van den Brandt; Gordon G A Hutchins; Susan D Richman; Tanwei Yuan; Rupert Langer; Josien C A Jenniskens; Kelly Offermans; Wolfram Mueller; Richard Gray; Stephen B Gruber; Joel K Greenson; Gad Rennert; Joseph D Bonner; Daniel Schmolze; Jitendra Jonnagaddala; Nicholas J Hawkins; Robyn L Ward; Dion Morton; Matthew Seymour; Laura Magill; Marta Nowak; Jennifer Hay; Viktor H Koelzer; David N Church; Christian Matek; Carol Geppert; Chaolong Peng; Cheng Zhi; Xiaoming Ouyang; Jacqueline A James; Maurice B Loughrey; Manuel Salto-Tellez; Hermann Brenner; Michael Hoffmeister; Daniel Truhn; Julia A Schnabel; Melanie Boxberg; Tingying Peng; Jakob Nikolas Kather; TransSCOT consortium

doi:10.1016/j.ccell.2023.08.002

Transformer-based biomarker prediction from colorectal cancer histology: A large-scale multicentric study

Sophia J Wagner, Daniel Reisenbüchler, Nicholas P West, Jan Moritz Niehues, Jiefu Zhu, Sebastian Foersch, Gregory Patrick Veldhuizen, Philip Quirke, Heike I Grabsch, Piet A van den Brandt, Gordon G A Hutchins, Susan D Richman, Tanwei Yuan, Rupert Langer, Josien C A Jenniskens, Kelly Offermans, Wolfram Mueller, Richard Gray, Stephen B Gruber, Joel K GreensonGad Rennert, Joseph D Bonner, Daniel Schmolze, Jitendra Jonnagaddala, Nicholas J Hawkins, Robyn L Ward, Dion Morton, Matthew Seymour, Laura Magill, Marta Nowak, Jennifer Hay, Viktor H Koelzer, David N Church, Christian Matek, Carol Geppert, Chaolong Peng, Cheng Zhi, Xiaoming Ouyang, Jacqueline A James, Maurice B Loughrey, Manuel Salto-Tellez, Hermann Brenner, Michael Hoffmeister, Daniel Truhn, Julia A Schnabel, Melanie Boxberg, Tingying Peng^*, Jakob Nikolas Kather^*, TransSCOT consortium

^*Corresponding author for this work

Research output: Contribution to journal › Article › Academic › peer-review

Abstract

Deep learning (DL) can accelerate the prediction of prognostic biomarkers from routine pathology slides in colorectal cancer (CRC). However, current approaches rely on convolutional neural networks (CNNs) and have mostly been validated on small patient cohorts. Here, we develop a new transformer-based pipeline for end-to-end biomarker prediction from pathology slides by combining a pre-trained transformer encoder with a transformer network for patch aggregation. Our transformer-based approach substantially improves the performance, generalizability, data efficiency, and interpretability as compared with current state-of-the-art algorithms. After training and evaluating on a large multicenter cohort of over 13,000 patients from 16 colorectal cancer cohorts, we achieve a sensitivity of 0.99 with a negative predictive value of over 0.99 for prediction of microsatellite instability (MSI) on surgical resection specimens. We demonstrate that resection specimen-only training reaches clinical-grade performance on endoscopic biopsy tissue, solving a long-standing diagnostic problem.

Original language	English
Pages (from-to)	1650-1661.e4
Number of pages	17
Journal	Cancer Cell
Volume	41
Issue number	9
DOIs	https://doi.org/10.1016/j.ccell.2023.08.002
Publication status	Published - 11 Sept 2023

Keywords

artificial intelligence
biomarker
colorectal cancer
deep learning
microsatellite instability
multiple instance learning
transformer

Access to Document

10.1016/j.ccell.2023.08.002Licence: CC BY

Cite this

Wagner, S. J., Reisenbüchler, D., West, N. P., Niehues, J. M., Zhu, J., Foersch, S., Veldhuizen, G. P., Quirke, P., Grabsch, H. I., van den Brandt, P. A., Hutchins, G. G. A., Richman, S. D., Yuan, T., Langer, R., Jenniskens, J. C. A., Offermans, K., Mueller, W., Gray, R., Gruber, S. B., ... TransSCOT consortium (2023). Transformer-based biomarker prediction from colorectal cancer histology: A large-scale multicentric study. Cancer Cell, 41(9), 1650-1661.e4. https://doi.org/10.1016/j.ccell.2023.08.002

@article{cff53291e926461983502cd8fed68a56,

title = "Transformer-based biomarker prediction from colorectal cancer histology: A large-scale multicentric study",

abstract = "Deep learning (DL) can accelerate the prediction of prognostic biomarkers from routine pathology slides in colorectal cancer (CRC). However, current approaches rely on convolutional neural networks (CNNs) and have mostly been validated on small patient cohorts. Here, we develop a new transformer-based pipeline for end-to-end biomarker prediction from pathology slides by combining a pre-trained transformer encoder with a transformer network for patch aggregation. Our transformer-based approach substantially improves the performance, generalizability, data efficiency, and interpretability as compared with current state-of-the-art algorithms. After training and evaluating on a large multicenter cohort of over 13,000 patients from 16 colorectal cancer cohorts, we achieve a sensitivity of 0.99 with a negative predictive value of over 0.99 for prediction of microsatellite instability (MSI) on surgical resection specimens. We demonstrate that resection specimen-only training reaches clinical-grade performance on endoscopic biopsy tissue, solving a long-standing diagnostic problem.",

keywords = "artificial intelligence, biomarker, colorectal cancer, deep learning, microsatellite instability, multiple instance learning, transformer",

author = "Wagner, {Sophia J} and Daniel Reisenb{\"u}chler and West, {Nicholas P} and Niehues, {Jan Moritz} and Jiefu Zhu and Sebastian Foersch and Veldhuizen, {Gregory Patrick} and Philip Quirke and Grabsch, {Heike I} and {van den Brandt}, {Piet A} and Hutchins, {Gordon G A} and Richman, {Susan D} and Tanwei Yuan and Rupert Langer and Jenniskens, {Josien C A} and Kelly Offermans and Wolfram Mueller and Richard Gray and Gruber, {Stephen B} and Greenson, {Joel K} and Gad Rennert and Bonner, {Joseph D} and Daniel Schmolze and Jitendra Jonnagaddala and Hawkins, {Nicholas J} and Ward, {Robyn L} and Dion Morton and Matthew Seymour and Laura Magill and Marta Nowak and Jennifer Hay and Koelzer, {Viktor H} and Church, {David N} and Christian Matek and Carol Geppert and Chaolong Peng and Cheng Zhi and Xiaoming Ouyang and James, {Jacqueline A} and Loughrey, {Maurice B} and Manuel Salto-Tellez and Hermann Brenner and Michael Hoffmeister and Daniel Truhn and Schnabel, {Julia A} and Melanie Boxberg and Tingying Peng and Kather, {Jakob Nikolas} and {TransSCOT consortium}",

year = "2023",

month = sep,

day = "11",

doi = "10.1016/j.ccell.2023.08.002",

language = "English",

volume = "41",

pages = "1650--1661.e4",

journal = "Cancer Cell",

issn = "1878-3686",

publisher = "Cell Press",

number = "9",

}

Wagner, SJ, Reisenbüchler, D, West, NP, Niehues, JM, Zhu, J, Foersch, S, Veldhuizen, GP, Quirke, P, Grabsch, HI , van den Brandt, PA, Hutchins, GGA, Richman, SD, Yuan, T, Langer, R, Jenniskens, JCA, Offermans, K, Mueller, W, Gray, R, Gruber, SB, Greenson, JK, Rennert, G, Bonner, JD, Schmolze, D, Jonnagaddala, J, Hawkins, NJ, Ward, RL, Morton, D, Seymour, M, Magill, L, Nowak, M, Hay, J, Koelzer, VH, Church, DN, Matek, C, Geppert, C, Peng, C, Zhi, C, Ouyang, X, James, JA, Loughrey, MB, Salto-Tellez, M, Brenner, H, Hoffmeister, M, Truhn, D, Schnabel, JA, Boxberg, M, Peng, T, Kather, JN & TransSCOT consortium 2023, 'Transformer-based biomarker prediction from colorectal cancer histology: A large-scale multicentric study', Cancer Cell, vol. 41, no. 9, pp. 1650-1661.e4. https://doi.org/10.1016/j.ccell.2023.08.002

TY - JOUR

T1 - Transformer-based biomarker prediction from colorectal cancer histology

T2 - A large-scale multicentric study

AU - Wagner, Sophia J

AU - Reisenbüchler, Daniel

AU - West, Nicholas P

AU - Niehues, Jan Moritz

AU - Zhu, Jiefu

AU - Foersch, Sebastian

AU - Veldhuizen, Gregory Patrick

AU - Quirke, Philip

AU - Grabsch, Heike I

AU - van den Brandt, Piet A

AU - Hutchins, Gordon G A

AU - Richman, Susan D

AU - Yuan, Tanwei

AU - Langer, Rupert

AU - Jenniskens, Josien C A

AU - Offermans, Kelly

AU - Mueller, Wolfram

AU - Gray, Richard

AU - Gruber, Stephen B

AU - Greenson, Joel K

AU - Rennert, Gad

AU - Bonner, Joseph D

AU - Schmolze, Daniel

AU - Jonnagaddala, Jitendra

AU - Hawkins, Nicholas J

AU - Ward, Robyn L

AU - Morton, Dion

AU - Seymour, Matthew

AU - Magill, Laura

AU - Nowak, Marta

AU - Hay, Jennifer

AU - Koelzer, Viktor H

AU - Church, David N

AU - Matek, Christian

AU - Geppert, Carol

AU - Peng, Chaolong

AU - Zhi, Cheng

AU - Ouyang, Xiaoming

AU - James, Jacqueline A

AU - Loughrey, Maurice B

AU - Salto-Tellez, Manuel

AU - Brenner, Hermann

AU - Hoffmeister, Michael

AU - Truhn, Daniel

AU - Schnabel, Julia A

AU - Boxberg, Melanie

AU - Peng, Tingying

AU - Kather, Jakob Nikolas

AU - TransSCOT consortium

PY - 2023/9/11

Y1 - 2023/9/11

N2 - Deep learning (DL) can accelerate the prediction of prognostic biomarkers from routine pathology slides in colorectal cancer (CRC). However, current approaches rely on convolutional neural networks (CNNs) and have mostly been validated on small patient cohorts. Here, we develop a new transformer-based pipeline for end-to-end biomarker prediction from pathology slides by combining a pre-trained transformer encoder with a transformer network for patch aggregation. Our transformer-based approach substantially improves the performance, generalizability, data efficiency, and interpretability as compared with current state-of-the-art algorithms. After training and evaluating on a large multicenter cohort of over 13,000 patients from 16 colorectal cancer cohorts, we achieve a sensitivity of 0.99 with a negative predictive value of over 0.99 for prediction of microsatellite instability (MSI) on surgical resection specimens. We demonstrate that resection specimen-only training reaches clinical-grade performance on endoscopic biopsy tissue, solving a long-standing diagnostic problem.

AB - Deep learning (DL) can accelerate the prediction of prognostic biomarkers from routine pathology slides in colorectal cancer (CRC). However, current approaches rely on convolutional neural networks (CNNs) and have mostly been validated on small patient cohorts. Here, we develop a new transformer-based pipeline for end-to-end biomarker prediction from pathology slides by combining a pre-trained transformer encoder with a transformer network for patch aggregation. Our transformer-based approach substantially improves the performance, generalizability, data efficiency, and interpretability as compared with current state-of-the-art algorithms. After training and evaluating on a large multicenter cohort of over 13,000 patients from 16 colorectal cancer cohorts, we achieve a sensitivity of 0.99 with a negative predictive value of over 0.99 for prediction of microsatellite instability (MSI) on surgical resection specimens. We demonstrate that resection specimen-only training reaches clinical-grade performance on endoscopic biopsy tissue, solving a long-standing diagnostic problem.

KW - artificial intelligence

KW - biomarker

KW - colorectal cancer

KW - deep learning

KW - microsatellite instability

KW - multiple instance learning

KW - transformer

U2 - 10.1016/j.ccell.2023.08.002

DO - 10.1016/j.ccell.2023.08.002

M3 - Article

SN - 1878-3686

VL - 41

SP - 1650-1661.e4

JO - Cancer Cell

JF - Cancer Cell

IS - 9

ER -

Transformer-based biomarker prediction from colorectal cancer histology: A large-scale multicentric study

Abstract

Keywords

Access to Document

Fingerprint

Cite this