Missing confounding information in counterfactual prediction models: a simulation study on model-based treatment effect evaluation in radiotherapy techniques

Background and purpose: Model-Based Clinical Evaluation (MBCE) uses counterfactual prediction to estimate causal treatment effects on radiation-induced toxicity reduction between radiotherapy techniques, such as proton versus photon therapy. In the Netherlands, patient selection for proton therapy follows a model-based approach using Normal Tissue Complication Probability (NTCP) models. These models prioritize prediction over causal inference and may potentially omit confounders influencing dose-toxicity relationships. This raises the question of whether NTCP models used for patient selection are suitable for MBCE. This study examines how omitted confounders in NTCP models and patient selection affect the validity of MBCE. Materials and methods: We simulated head and neck photon therapy patients while varying a confounder's effect on radiation dose and toxicity. Model-based selection for proton therapy followed the Dutch National Indication Protocol. The average treatment effect in the proton-treated group (ATT) was estimated using a current NTCP model (excluding the confounder) and an extended NTCP model (including the confounder), then compared to the true effect. Additional simulations explored modified patient-selection scenarios. Results: In the primary simulation, the current model produced unbiased ATT estimates, because the omitted confounder was conditionally independent of patient selection. When this assumption was violated, i.e., when the omitted predictor was associated with patient-selection, the current model introduced bias while the extended model did not. Conclusion: Omitting confounders in NTCP models for MBCE does not inherently bias MBCE estimates. However, its validity depends on whether omitted predictors are associated with the patient-selection mechanism. We recommend including outcome predictors related to treatment allocation in NTCP models used for MBCE.