Validation of models to diagnose ovarian cancer in patients managed surgically or conservatively: multicentre cohort study

Ben Van Calster; Lil Valentin; Wouter Froyman; Chiara Landolfo; Jolien Ceusters; Antonia C. Testa; Laure Wynants; Povilas Sladkevicius; Caroline Van Holsbeke; Ekaterini Domali; Robert Fruscio; Elisabeth Epstein; Dorella Franchi; Marek J. Kudla; Valentina Chiappa; Juan L. Alcazar; Francesco P. G. Leone; Francesca Buonomo; Maria Elisabetta Coccia; Stefano Guerriero; Nandita Deo; Ligita Jokubkiene; Luca Savelli; Daniela Fischerova; Artur Czekierdowski; Jeroen Kaijser; An Coosemans; Giovanni Scambia; Ignace Vergote; Tom Bourne; Dirk Timmerman

doi:10.1136/bmj.m2614

Validation of models to diagnose ovarian cancer in patients managed surgically or conservatively: multicentre cohort study

Ben Van Calster, Lil Valentin, Wouter Froyman, Chiara Landolfo, Jolien Ceusters, Antonia C. Testa, Laure Wynants, Povilas Sladkevicius, Caroline Van Holsbeke, Ekaterini Domali, Robert Fruscio, Elisabeth Epstein, Dorella Franchi, Marek J. Kudla, Valentina Chiappa, Juan L. Alcazar, Francesco P. G. Leone, Francesca Buonomo, Maria Elisabetta Coccia, Stefano GuerrieroNandita Deo, Ligita Jokubkiene, Luca Savelli, Daniela Fischerova, Artur Czekierdowski, Jeroen Kaijser, An Coosemans, Giovanni Scambia, Ignace Vergote, Tom Bourne, Dirk Timmerman^*

^*Corresponding author for this work

Research output: Contribution to journal › Article › Academic › peer-review

Abstract

OBJECTIVE

To evaluate the performance of diagnostic prediction models for ovarian malignancy in all patients with an ovarian mass managed surgically or conservatively.

DESIGN

Multicentre cohort study.

SETTING

36 oncology referral centres (tertiary centres with a specific gynaecological oncology unit) or other types of centre.

PARTICIPANTS

Consecutive adult patients presenting with an adnexal mass between January 2012 and March 2015 and managed by surgery or follow-up.

MAIN OUTCOME MEASURES

Overall and centre specific discrimination, calibration, and clinical utility of six prediction models for ovarian malignancy (risk of malignancy index (RMI), logistic regression model 2 (LR2), simple rules, simple rules risk model (SRRisk), assessment of different neoplasias in the adnexa (ADNEX) with or without CA125). ADNEX allows the risk of malignancy to be subdivided into risks of a borderline, stage I primary, stage II-IV primary, or secondary metastatic malignancy. The outcome was based on histology if patients underwent surgery, or on results of clinical and ultrasound follow-up at 12 (+/- 2) months. Multiple imputation was used when outcome based on follow-up was uncertain.

RESULTS

The primary analysis included 17 centres that met strict quality criteria for surgical and follow-up data (5717 of all 8519 patients). 812 patients (14%) had a mass that was already in follow-up at study recruitment, therefore 4905 patients were included in the statistical analysis. The outcome was benign in 3441 (70%) patients and malignant in 978 (20%). Uncertain outcomes (486, 10%) were most often explained by limited follow-up information. The overall area under the receiver operating characteristic curve was highest for ADNEX with CA125 (0.94, 95% confidence interval 0.92 to 0.96), ADNEX without CA125 (0.94, 0.91 to 0.95) and SRRisk (0.94, 0.91 to 0.95), and lowest for RMI (0.89, 0.85 to 0.92). Calibration varied among centres for all models, however the ADNEX models and SRRisk were the best calibrated. Calibration of the estimated risks for the tumour subtypes was good for ADNEX irrespective of whether or not CA125 was included as a predictor. Overall clinical utility (net benefit) was highest for the ADNEX models and SRRisk, and lowest for RMI. For patients who received at least one follow-up scan (n=1958), overall area under the receiver operating characteristic curve ranged from 0.76 (95% confidence interval 0.66 to 0.84) for RMI to 0.89 (0.81 to 0.94) for ADNEX with CA125.

CONCLUSIONS

Our study found the ADNEX models and SRRisk are the best models to distinguish between benign and malignant masses in all patients presenting with an adnexal mass, including those managed conservatively.

Original language	English
Article number	m2614
Number of pages	12
Journal	BMJ
Volume	370
DOIs	https://doi.org/10.1136/bmj.m2614
Publication status	Published - 30 Jul 2020

Keywords

IOTA ADNEX MODEL
EXTERNAL VALIDATION
RISK MODELS
ULTRASOUND
MASSES
MALIGNANCY
SURGERY
PREDICTION
TUMORS
CALIBRATION

Access to Document

10.1136/bmj.m2614Licence: CC BY

Cite this

Van Calster, B., Valentin, L., Froyman, W., Landolfo, C., Ceusters, J., Testa, A. C., Wynants, L., Sladkevicius, P., Van Holsbeke, C., Domali, E., Fruscio, R., Epstein, E., Franchi, D., Kudla, M. J., Chiappa, V., Alcazar, J. L., Leone, F. P. G., Buonomo, F., Coccia, M. E., ... Timmerman, D. (2020). Validation of models to diagnose ovarian cancer in patients managed surgically or conservatively: multicentre cohort study. BMJ, 370, Article m2614. https://doi.org/10.1136/bmj.m2614

@article{20432a1630e447d6a802afea06b9f7ec,

title = "Validation of models to diagnose ovarian cancer in patients managed surgically or conservatively: multicentre cohort study",

abstract = "OBJECTIVETo evaluate the performance of diagnostic prediction models for ovarian malignancy in all patients with an ovarian mass managed surgically or conservatively.DESIGNMulticentre cohort study.SETTING36 oncology referral centres (tertiary centres with a specific gynaecological oncology unit) or other types of centre.PARTICIPANTSConsecutive adult patients presenting with an adnexal mass between January 2012 and March 2015 and managed by surgery or follow-up.MAIN OUTCOME MEASURESOverall and centre specific discrimination, calibration, and clinical utility of six prediction models for ovarian malignancy (risk of malignancy index (RMI), logistic regression model 2 (LR2), simple rules, simple rules risk model (SRRisk), assessment of different neoplasias in the adnexa (ADNEX) with or without CA125). ADNEX allows the risk of malignancy to be subdivided into risks of a borderline, stage I primary, stage II-IV primary, or secondary metastatic malignancy. The outcome was based on histology if patients underwent surgery, or on results of clinical and ultrasound follow-up at 12 (+/- 2) months. Multiple imputation was used when outcome based on follow-up was uncertain.RESULTSThe primary analysis included 17 centres that met strict quality criteria for surgical and follow-up data (5717 of all 8519 patients). 812 patients (14%) had a mass that was already in follow-up at study recruitment, therefore 4905 patients were included in the statistical analysis. The outcome was benign in 3441 (70%) patients and malignant in 978 (20%). Uncertain outcomes (486, 10%) were most often explained by limited follow-up information. The overall area under the receiver operating characteristic curve was highest for ADNEX with CA125 (0.94, 95% confidence interval 0.92 to 0.96), ADNEX without CA125 (0.94, 0.91 to 0.95) and SRRisk (0.94, 0.91 to 0.95), and lowest for RMI (0.89, 0.85 to 0.92). Calibration varied among centres for all models, however the ADNEX models and SRRisk were the best calibrated. Calibration of the estimated risks for the tumour subtypes was good for ADNEX irrespective of whether or not CA125 was included as a predictor. Overall clinical utility (net benefit) was highest for the ADNEX models and SRRisk, and lowest for RMI. For patients who received at least one follow-up scan (n=1958), overall area under the receiver operating characteristic curve ranged from 0.76 (95% confidence interval 0.66 to 0.84) for RMI to 0.89 (0.81 to 0.94) for ADNEX with CA125.CONCLUSIONSOur study found the ADNEX models and SRRisk are the best models to distinguish between benign and malignant masses in all patients presenting with an adnexal mass, including those managed conservatively.",

keywords = "IOTA ADNEX MODEL, EXTERNAL VALIDATION, RISK MODELS, ULTRASOUND, MASSES, MALIGNANCY, SURGERY, PREDICTION, TUMORS, CALIBRATION",

author = "{Van Calster}, Ben and Lil Valentin and Wouter Froyman and Chiara Landolfo and Jolien Ceusters and Testa, {Antonia C.} and Laure Wynants and Povilas Sladkevicius and {Van Holsbeke}, Caroline and Ekaterini Domali and Robert Fruscio and Elisabeth Epstein and Dorella Franchi and Kudla, {Marek J.} and Valentina Chiappa and Alcazar, {Juan L.} and Leone, {Francesco P. G.} and Francesca Buonomo and Coccia, {Maria Elisabetta} and Stefano Guerriero and Nandita Deo and Ligita Jokubkiene and Luca Savelli and Daniela Fischerova and Artur Czekierdowski and Jeroen Kaijser and An Coosemans and Giovanni Scambia and Ignace Vergote and Tom Bourne and Dirk Timmerman",

note = "Publisher Copyright: {\textcopyright} Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY. No commercial re-use. See rights and permissions. Published by BMJ.",

year = "2020",

month = jul,

day = "30",

doi = "10.1136/bmj.m2614",

language = "English",

volume = "370",

journal = "BMJ",

issn = "1756-1833",

publisher = "BMJ Publishing Group",

}

Van Calster, B, Valentin, L, Froyman, W, Landolfo, C, Ceusters, J, Testa, AC, Wynants, L, Sladkevicius, P, Van Holsbeke, C, Domali, E, Fruscio, R, Epstein, E, Franchi, D, Kudla, MJ, Chiappa, V, Alcazar, JL, Leone, FPG, Buonomo, F, Coccia, ME, Guerriero, S, Deo, N, Jokubkiene, L, Savelli, L, Fischerova, D, Czekierdowski, A, Kaijser, J, Coosemans, A, Scambia, G, Vergote, I, Bourne, T & Timmerman, D 2020, 'Validation of models to diagnose ovarian cancer in patients managed surgically or conservatively: multicentre cohort study', BMJ, vol. 370, m2614. https://doi.org/10.1136/bmj.m2614

TY - JOUR

T1 - Validation of models to diagnose ovarian cancer in patients managed surgically or conservatively

T2 - multicentre cohort study

AU - Van Calster, Ben

AU - Valentin, Lil

AU - Froyman, Wouter

AU - Landolfo, Chiara

AU - Ceusters, Jolien

AU - Testa, Antonia C.

AU - Wynants, Laure

AU - Sladkevicius, Povilas

AU - Van Holsbeke, Caroline

AU - Domali, Ekaterini

AU - Fruscio, Robert

AU - Epstein, Elisabeth

AU - Franchi, Dorella

AU - Kudla, Marek J.

AU - Chiappa, Valentina

AU - Alcazar, Juan L.

AU - Leone, Francesco P. G.

AU - Buonomo, Francesca

AU - Coccia, Maria Elisabetta

AU - Guerriero, Stefano

AU - Deo, Nandita

AU - Jokubkiene, Ligita

AU - Savelli, Luca

AU - Fischerova, Daniela

AU - Czekierdowski, Artur

AU - Kaijser, Jeroen

AU - Coosemans, An

AU - Scambia, Giovanni

AU - Vergote, Ignace

AU - Bourne, Tom

AU - Timmerman, Dirk

PY - 2020/7/30

Y1 - 2020/7/30

N2 - OBJECTIVETo evaluate the performance of diagnostic prediction models for ovarian malignancy in all patients with an ovarian mass managed surgically or conservatively.DESIGNMulticentre cohort study.SETTING36 oncology referral centres (tertiary centres with a specific gynaecological oncology unit) or other types of centre.PARTICIPANTSConsecutive adult patients presenting with an adnexal mass between January 2012 and March 2015 and managed by surgery or follow-up.MAIN OUTCOME MEASURESOverall and centre specific discrimination, calibration, and clinical utility of six prediction models for ovarian malignancy (risk of malignancy index (RMI), logistic regression model 2 (LR2), simple rules, simple rules risk model (SRRisk), assessment of different neoplasias in the adnexa (ADNEX) with or without CA125). ADNEX allows the risk of malignancy to be subdivided into risks of a borderline, stage I primary, stage II-IV primary, or secondary metastatic malignancy. The outcome was based on histology if patients underwent surgery, or on results of clinical and ultrasound follow-up at 12 (+/- 2) months. Multiple imputation was used when outcome based on follow-up was uncertain.RESULTSThe primary analysis included 17 centres that met strict quality criteria for surgical and follow-up data (5717 of all 8519 patients). 812 patients (14%) had a mass that was already in follow-up at study recruitment, therefore 4905 patients were included in the statistical analysis. The outcome was benign in 3441 (70%) patients and malignant in 978 (20%). Uncertain outcomes (486, 10%) were most often explained by limited follow-up information. The overall area under the receiver operating characteristic curve was highest for ADNEX with CA125 (0.94, 95% confidence interval 0.92 to 0.96), ADNEX without CA125 (0.94, 0.91 to 0.95) and SRRisk (0.94, 0.91 to 0.95), and lowest for RMI (0.89, 0.85 to 0.92). Calibration varied among centres for all models, however the ADNEX models and SRRisk were the best calibrated. Calibration of the estimated risks for the tumour subtypes was good for ADNEX irrespective of whether or not CA125 was included as a predictor. Overall clinical utility (net benefit) was highest for the ADNEX models and SRRisk, and lowest for RMI. For patients who received at least one follow-up scan (n=1958), overall area under the receiver operating characteristic curve ranged from 0.76 (95% confidence interval 0.66 to 0.84) for RMI to 0.89 (0.81 to 0.94) for ADNEX with CA125.CONCLUSIONSOur study found the ADNEX models and SRRisk are the best models to distinguish between benign and malignant masses in all patients presenting with an adnexal mass, including those managed conservatively.

AB - OBJECTIVETo evaluate the performance of diagnostic prediction models for ovarian malignancy in all patients with an ovarian mass managed surgically or conservatively.DESIGNMulticentre cohort study.SETTING36 oncology referral centres (tertiary centres with a specific gynaecological oncology unit) or other types of centre.PARTICIPANTSConsecutive adult patients presenting with an adnexal mass between January 2012 and March 2015 and managed by surgery or follow-up.MAIN OUTCOME MEASURESOverall and centre specific discrimination, calibration, and clinical utility of six prediction models for ovarian malignancy (risk of malignancy index (RMI), logistic regression model 2 (LR2), simple rules, simple rules risk model (SRRisk), assessment of different neoplasias in the adnexa (ADNEX) with or without CA125). ADNEX allows the risk of malignancy to be subdivided into risks of a borderline, stage I primary, stage II-IV primary, or secondary metastatic malignancy. The outcome was based on histology if patients underwent surgery, or on results of clinical and ultrasound follow-up at 12 (+/- 2) months. Multiple imputation was used when outcome based on follow-up was uncertain.RESULTSThe primary analysis included 17 centres that met strict quality criteria for surgical and follow-up data (5717 of all 8519 patients). 812 patients (14%) had a mass that was already in follow-up at study recruitment, therefore 4905 patients were included in the statistical analysis. The outcome was benign in 3441 (70%) patients and malignant in 978 (20%). Uncertain outcomes (486, 10%) were most often explained by limited follow-up information. The overall area under the receiver operating characteristic curve was highest for ADNEX with CA125 (0.94, 95% confidence interval 0.92 to 0.96), ADNEX without CA125 (0.94, 0.91 to 0.95) and SRRisk (0.94, 0.91 to 0.95), and lowest for RMI (0.89, 0.85 to 0.92). Calibration varied among centres for all models, however the ADNEX models and SRRisk were the best calibrated. Calibration of the estimated risks for the tumour subtypes was good for ADNEX irrespective of whether or not CA125 was included as a predictor. Overall clinical utility (net benefit) was highest for the ADNEX models and SRRisk, and lowest for RMI. For patients who received at least one follow-up scan (n=1958), overall area under the receiver operating characteristic curve ranged from 0.76 (95% confidence interval 0.66 to 0.84) for RMI to 0.89 (0.81 to 0.94) for ADNEX with CA125.CONCLUSIONSOur study found the ADNEX models and SRRisk are the best models to distinguish between benign and malignant masses in all patients presenting with an adnexal mass, including those managed conservatively.

KW - IOTA ADNEX MODEL

KW - EXTERNAL VALIDATION

KW - RISK MODELS

KW - ULTRASOUND

KW - MASSES

KW - MALIGNANCY

KW - SURGERY

KW - PREDICTION

KW - TUMORS

KW - CALIBRATION

U2 - 10.1136/bmj.m2614

DO - 10.1136/bmj.m2614

M3 - Article

C2 - 32732303

SN - 1756-1833

VL - 370

JO - BMJ

JF - BMJ

M1 - m2614

ER -