A Framework for Variational Inference of Lightweight Bayesian Neural
  Networks with Heteroscedastic Uncertainties
David J. Schodt Ryan Brown Michael Merritt Samuel Park Delsin Menolascino Mark A. Peot
Obtaining heteroscedastic predictive uncertainties from a Bayesian Neural
Network (BNN) is vital to many applications. Often, heteroscedastic aleatoric
uncertainties are learned as outputs of the BNN in addition to the predictive
means, however doing so may necessitate adding more learnable parameters to the
network. In this work, we demonstrate that both the heteroscedastic aleatoric
and epistemic variance can be embedded into the variances of learned BNN
parameters, improving predictive performance for lightweight networks. By
complementing this approach with a moment propagation approach to inference, we
introduce a relatively simple framework for sampling-free variational inference
suitable for lightweight BNNs.