External validation of deep learning-based contouring of head and neck organs at risk

Ellen J L Brunenberg*, Isabell K Steinseifer, Sven van den Bosch, Johannes H A M Kaanders, Charlotte L Brouwer, Mark J Gooding, Wouter van Elmpt, Rene Monshouwer

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review


Background and purpose: Head and neck (HN) radiotherapy can benefit from automatic delineation of tumor and surrounding organs because of the complex anatomy and the regular need for adaptation. The aim of this study was to assess the performance of a commercially available deep learning contouring (DLC) model on an external validation set.

Materials and methods: The CT-based DLC model, trained at the University Medical Center Groningen (UMCG), was applied to an independent set of 58 patients from the Radboud University Medical Center (RUMC). DLC results were compared to the RUMC manual reference using the Dice similarity coefficient (DSC) and 95th percentile of Hausdorff distance (HD95). Craniocaudal spatial information was added by calculating binned measures. In addition, a qualitative evaluation compared the acceptance of manual and DLC contours in both groups of observers.

Results: Good correspondence was shown for the mandible (DSC 0.90; HD95 3.6 mm). Performance was reasonable for the glandular OARs, brainstem and oral cavity (DSC 0.78-0.85, HD95 3.7-7.3 mm). The other aerodigestive tract OARs showed only moderate agreement (DSC 0.53-0.65, HD95 around 9 mm). The binned measures displayed the largest deviations caudally and/or cranially.

Conclusions: This study demonstrates that the DLC model can provide a reasonable starting point for delineation when applied to an independent patient cohort. The qualitative evaluation did not reveal large differences in the interpretation of contouring guidelines between RUMC and UMCG observers.

Original languageEnglish
Pages (from-to)8-15
Number of pages8
JournalPhysics & Imaging in Radiation Oncology
Publication statusPublished - Jul 2020


  • Deep learning
  • Auto-contouring
  • Contour comparison
  • Head & neck cancer
  • CT


Dive into the research topics of 'External validation of deep learning-based contouring of head and neck organs at risk'. Together they form a unique fingerprint.

Cite this