Posterior concentrations of fully-connected Bayesian neural networks
  with general priors on the weights
Insung Kong Yongdai Kim
Bayesian approaches for training deep neural networks (BNNs) have received
significant interest and have been effectively utilized in a wide range of
applications. There have been several studies on the properties of posterior
concentrations of BNNs. However, most of these studies only demonstrate results
in BNN models with sparse or heavy-tailed priors. Surprisingly, no theoretical
results currently exist for BNNs using Gaussian priors, which are the most
commonly used one. The lack of theory arises from the absence of approximation
results of Deep Neural Networks (DNNs) that are non-sparse and have bounded
parameters. In this paper, we present a new approximation theory for non-sparse
DNNs with bounded parameters. Additionally, based on the approximation theory,
we show that BNNs with non-sparse general priors can achieve near-minimax
optimal posterior concentration rates to the true model.