Efficient Conformal Prediction under Data Heterogeneity
Vincent Plassier Nikita Kotelevskii Aleksandr Rubashevskii Fedor Noskov Maksim Velikanov Alexander Fishkov Samuel Horvath Martin Takac Eric Moulines Maxim Panov
Conformal Prediction (CP) stands out as a robust framework for uncertainty
quantification, which is crucial for ensuring the reliability of predictions.
However, common CP methods heavily rely on data exchangeability, a condition
often violated in practice. Existing approaches for tackling
non-exchangeability lead to methods that are not computable beyond the simplest
examples. This work introduces a new efficient approach to CP that produces
provably valid confidence sets for fairly general non-exchangeable data
distributions. We illustrate the general theory with applications to the
challenging setting of federated learning under data heterogeneity between
agents. Our method allows constructing provably valid personalized prediction
sets for agents in a fully federated way. The effectiveness of the proposed
method is demonstrated in a series of experiments on real-world datasets.