Multi-Modal Conformal Prediction Regions by Optimizing Convex Shape
  Templates
Renukanandan Tumu Matthew Cleaveland Rahul Mangharam George J. Pappas Lars Lindemann
Conformal prediction is a statistical tool for producing prediction regions
for machine learning models that are valid with high probability. A key
component of conformal prediction algorithms is a non-conformity score function
that quantifies how different a model's prediction is from the unknown ground
truth value. Essentially, these functions determine the shape and the size of
the conformal prediction regions. However, little work has gone into finding
non-conformity score functions that produce prediction regions that are
multi-modal and practical, i.e., that can efficiently be used in engineering
applications. We propose a method that optimizes parameterized shape template
functions over calibration data, which results in non-conformity score
functions that produce prediction regions with minimum volume. Our approach
results in prediction regions that are multi-modal, so they can properly
capture residuals of distributions that have multiple modes, and practical, so
each region is convex and can be easily incorporated into downstream tasks,
such as a motion planner using conformal prediction regions. Our method applies
to general supervised learning tasks, while we illustrate its use in
time-series prediction. We provide a toolbox and present illustrative case
studies of F16 fighter jets and autonomous vehicles, showing an up to $68\%$
reduction in prediction region area.