Self-Consistent Conformal Prediction
Lars van der Laan Ahmed M. Alaa
In decision-making guided by machine learning, decision-makers often take
identical actions in contexts with identical predicted outcomes. Conformal
prediction helps decision-makers quantify outcome uncertainty for actions,
allowing for better risk management. Inspired by this perspective, we introduce
self-consistent conformal prediction, which yields both Venn-Abers calibrated
predictions and conformal prediction intervals that are valid conditional on
actions prompted by model predictions. Our procedure can be applied post-hoc to
any black-box predictor to provide rigorous, action-specific decision-making
guarantees. Numerical experiments show our approach strikes a balance between
interval efficiency and conditional validity.