Clinically inappropriate post hoc exclusion of study participants from test accuracy calculations: the ROMA score, an example from a recent NICE diagnostic assessment

Shona Lang, Nigel Armstrong, Sohan Deshpande, Bram Ramaekers, Sabine Grimm, Shelley de Kock, Jos Kleijnen, Marie Westwood*

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

Objective To explore how the definition of the target condition and post hoc exclusion of participants can limit the usefulness of diagnostic accuracy studies. Methods We used data from a systematic review, conducted for a NICE diagnostic assessment of risk scores to inform secondary care decisions about specialist referral for women with suspected ovarian cancer, to explore how the definition of the target condition and post hoc exclusion of participants can limit the usefulness of diagnostic accuracy studies to inform clinical practice. Results Fourteen of the studies evaluated the ROMA score, nine used Abbott ARCHITECT tumour marker assays, five used Roche Elecsys. The summary sensitivity estimate (Abbott ARCHITECT) was highest, 95.1% (95% CI: 92.4 to 97.1%), where analyses excluded participants with borderline tumours or malignancies other than epithelial ovarian cancer and lowest, 75.0% (95% CI: 60.4 to 86.4%), where all participants were included. Results were similar for Roche Elecsys tumour marker assays. Although the number of patients involved was small, data from studies that reported diagnostic accuracy for both the whole study population and with post hoc exclusion of those with borderline or non-epithelial malignancies suggested that patients with borderline or malignancies other than epithelial ovarian cancer accounts for between 50 and 85% of false-negative ROMA scores. Conclusions Our results illustrate the potential consequences of inappropriate population selection in diagnostic studies; women with non-epithelial ovarian cancers or non-ovarian primaries, and those borderline tumours may be disproportionately represented among those with false negative, 'low risk' ROMA scores. These observations highlight the importance of giving careful consideration to how the target condition has been defined when assessing whether the diagnostic accuracy estimates reported in clinical studies will translate into clinical utility in real-world settings.

Original languageEnglish
Pages (from-to)72-81
Number of pages10
JournalAnnals of Clinical Biochemistry
Volume56
Issue number1
DOIs
Publication statusPublished - 1 Jan 2019

Keywords

  • CA125
  • CANCER DETECTION
  • CUTOFF VALUES
  • DIFFERENTIAL-DIAGNOSIS
  • HE-4
  • INDEX
  • OVARIAN MALIGNANCY ALGORITHM
  • PREOPERATIVE DIAGNOSIS
  • RISK
  • Tumour markers
  • WOMEN
  • cancer
  • statistics

Cite this