Extending ROC Analysis to Uncertainty-Aware Risk Prediction with an Interval-Based AUC (iAUC)

In high-stakes risk prediction, interval-valued predictions provide a natural way to represent predictive uncertainty. However, standard evaluation tools such as the receiver operating characteristic (ROC) curve and the area under the curve (AUC) are designed for point-valued predictions and do not capture how uncertainty affects discrimination performance. To address this gap, we develop an uncertainty-aware ROC framework and corresponding interval-based AUC (iAUC) metrics for evaluating the performance of interval-valued risk predictions. The framework constructs two ROC-style curves with associated areas $\mathrm{AUC}_L$ and $\mathrm{AUC}_U$, and shows that these quantities induce a three-region decomposition of positive-negative rankings into confidently correct, ambiguous, and confidently incorrect orderings. Under valid class-conditional coverage, $\mathrm{AUC}_L$ and $\mathrm{AUC}_U$, together with the pairwise miscoverage rate, provide theoretical bounds on the optimal AUC, linking interval coverage to achievable discrimination. The framework is compatible with a range of methods for quantifying epistemic uncertainty in risk prediction, including Bayesian, bootstrap, and ensemble-based procedures, and provides a practical way to compare predictive models and interval-generation methods. Experiments on clinical benchmark data and MIMIC-IV mortality prediction show that models with similar point AUC can have different uncertainty-aware ranking profiles, and that the proposed decomposition can support risk-aware model selection and threshold-based abstention.