Conformalized Deep Splines for Optimal and Efficient Prediction Sets
Nathaniel Diamant Ehsan Hajiramezanali Tommaso Biancalani Gabriele Scalia
Uncertainty estimation is critical in high-stakes machine learning
applications. One effective way to estimate uncertainty is conformal
prediction, which can provide predictive inference with statistical coverage
guarantees. We present a new conformal regression method, Spline Prediction
Intervals via Conformal Estimation (SPICE), that estimates the conditional
density using neural-network-parameterized splines. We prove universal
approximation and optimality results for SPICE, which are empirically validated
by our experiments. SPICE is compatible with two different efficient-to-compute
conformal scores, one oracle-optimal for marginal coverage (SPICE-ND) and the
other asymptotically optimal for conditional coverage (SPICE-HPD). Results on
benchmark datasets demonstrate SPICE-ND models achieve the smallest average
prediction set sizes, including average size reductions of nearly 50% for some
datasets compared to the next best baseline. SPICE-HPD models achieve the best
conditional coverage compared to baselines. The SPICE implementation is made
available.