Testing Marginal and Conditional Coverage in Conformal Prediction for Non-Stationary Time Series via Value-at-Risk Backtesting

Conformal Prediction (CP) constructs prediction intervals with marginal coverage guarantees under the assumption of exchangeability, yet it has also been widely applied to non-exchangeable settings such as time series, where temporal dependence and distribution shifts often violate this assumption. Despite this, CP methods are typically evaluated using descriptive metrics like empirical coverage and average interval width, without formal statistical testing. This lack of hypothesis-driven evaluation makes it unclear whether deviations are meaningful or due to random variation. We address this gap by establishing a formal equivalence between CP and Value at Risk (VaR), enabling the use of VaR-style backtesting methods to statistically assess both marginal and conditional coverage. Additionally, we incorporate Diebold-Mariano tests with interval scores to compare predictive performance. Applied to synthetic, electricity, and financial time series, our framework uncovers violation and adaptation issues overlooked by standard metrics. The Dynamic Binary Test and Geometric Conformal Backtesting, in particular, identify covariate- and drift-induced dependence and miscalibration, offering a sharper lens for evaluating CP methods in non-stationary settings.