Model-free generalized fiducial inference
Jonathan P Williams
Motivated by the need for the development of safe and reliable methods for
uncertainty quantification in machine learning, I propose and develop ideas for
a model-free statistical framework for imprecise probabilistic prediction
inference. This framework facilitates uncertainty quantification in the form of
prediction sets that offer finite sample control of type 1 errors, a property
shared with conformal prediction sets, but this new approach also offers more
versatile tools for imprecise probabilistic reasoning. Furthermore, I propose
and consider the theoretical and empirical properties of a precise
probabilistic approximation to the model-free imprecise framework.
Approximating a belief/plausibility measure pair by an [optimal in some sense]
probability measure in the credal set is a critical resolution needed for the
broader adoption of imprecise probabilistic approaches to inference in
statistical and machine learning communities. It is largely undetermined in the
statistical and machine learning literatures, more generally, how to properly
quantify uncertainty in that there is no generally accepted standard of
accountability of stated uncertainties. The research I present in this
manuscript is aimed at motivating a framework for statistical inference with
reliability and accountability as the guiding principles.