TY - JOUR
T1 - Evaluating the human-artificial intelligence interaction in optical diagnosis of early colorectal carcinoma through endoscopist responses to simulated artificial intelligence
T2 - a multinational online study
AU - Thijssen, Ayla
AU - Schreuder, Ramon-Michel
AU - Dehghani, Nikoo
AU - Dekker, Evelien
AU - Boonstra, Jurjen J.
AU - Schrauwen, Ruud W. M.
AU - Baven-Pronk, Martine A. M. C.
AU - Terhaar sive Droste, Jochim S.
AU - Bos, Philip R.
AU - de Vos tot Nederveen Cappel, Wouter H.
AU - Hadithi, Muhammed
AU - Straathof, Jan-Willem A.
AU - Bulte, Geert J.
AU - Bastiaansen, Barbara A. J.
AU - Hoge, Chantal
AU - Perk, Lars E.
AU - Groen, John N.
AU - ter Borg, Frank
AU - van Leerdam, Monique E.
AU - Rondagh, Eveline J. A.
AU - de With, Peter H. N.
AU - Smulders, Fren T. Y.
AU - Winkens, Bjorn
AU - Moons, Leon M. G.
AU - van der Sommen, Fons
AU - Schoon, Erik J.
AU - Dutch T1 Colorectal Cancer Working Group
AU - COMET T1 CRC study group
PY - 2026/8
Y1 - 2026/8
N2 - Background Suboptimal human-artificial intelligence (AI) interaction is a potential roadblock for implementation of AI in clinical practice. We aimed to evaluate the interaction between AI and endoscopists in the optical diagnosis of colorectal carcinoma (CRC). Methods International endoscopists were invited to diagnose colorectal lesions online. After a pretest of 15 videos, the diagnosis could be adjusted in the subsequent 35 videos after receiving a simulated AI diagnosis. Histology was the gold standard. Endoscopists’ diagnoses before and after AI were compared using the signal detection theory. Inappropriate response to AI was defined as decreased diagnostic performance by the endoscopist after AI. Results In 2730 diagnoses, the 78 participating endoscopists reached diagnostic accuracies of 71.4 % before and 73.7 % after AI (P < 0.001). Before AI, endoscopists missed only a few true CRCs (mean hit rate 0.86), at the cost of a mean false alarm rate of 0.35. Inappropriate response to AI was observed in 28.2 % of endoscopists. After AI, most endoscopists (41.0 %) shifted toward a more conservative attitude, tending to diagnose a lesion as non-CRC. Conclusions Inappropriate response to AI is an issue in human-AI interaction for the optical diagnosis of CRC. Before using AI, endoscopists should become familiar with their own optical diagnostic performance, and that of the AI system, to possibly improve interaction.
AB - Background Suboptimal human-artificial intelligence (AI) interaction is a potential roadblock for implementation of AI in clinical practice. We aimed to evaluate the interaction between AI and endoscopists in the optical diagnosis of colorectal carcinoma (CRC). Methods International endoscopists were invited to diagnose colorectal lesions online. After a pretest of 15 videos, the diagnosis could be adjusted in the subsequent 35 videos after receiving a simulated AI diagnosis. Histology was the gold standard. Endoscopists’ diagnoses before and after AI were compared using the signal detection theory. Inappropriate response to AI was defined as decreased diagnostic performance by the endoscopist after AI. Results In 2730 diagnoses, the 78 participating endoscopists reached diagnostic accuracies of 71.4 % before and 73.7 % after AI (P < 0.001). Before AI, endoscopists missed only a few true CRCs (mean hit rate 0.86), at the cost of a mean false alarm rate of 0.35. Inappropriate response to AI was observed in 28.2 % of endoscopists. After AI, most endoscopists (41.0 %) shifted toward a more conservative attitude, tending to diagnose a lesion as non-CRC. Conclusions Inappropriate response to AI is an issue in human-AI interaction for the optical diagnosis of CRC. Before using AI, endoscopists should become familiar with their own optical diagnostic performance, and that of the AI system, to possibly improve interaction.
U2 - 10.1055/a-2912-9478
DO - 10.1055/a-2912-9478
M3 - Article
SN - 0013-726X
JO - Endoscopy
JF - Endoscopy
ER -