Morse Neural Networks for Uncertainty Quantification
Benoit Dherin Huiyi Hu Jie Ren Michael W. Dusenberry Balaji Lakshminarayanan
We introduce a new deep generative model useful for uncertainty
quantification: the Morse neural network, which generalizes the unnormalized
Gaussian densities to have modes of high-dimensional submanifolds instead of
just discrete points. Fitting the Morse neural network via a KL-divergence loss
yields 1) a (unnormalized) generative density, 2) an OOD detector, 3) a
calibration temperature, 4) a generative sampler, along with in the supervised
case 5) a distance aware-classifier. The Morse network can be used on top of a
pre-trained network to bring distance-aware calibration w.r.t the training
data. Because of its versatility, the Morse neural networks unifies many
techniques: e.g., the Entropic Out-of-Distribution Detector of (Mac\^edo et
al., 2021) in OOD detection, the one class Deep Support Vector Description
method of (Ruff et al., 2018) in anomaly detection, or the Contrastive One
Class classifier in continuous learning (Sun et al., 2021). The Morse neural
network has connections to support vector machines, kernel methods, and Morse
theory in topology.