Bayesian Domain Invariant Learning via Posterior Generalization of
  Parameter Distributions
Shiyu Shen Bin Pan Tianyang Shi Tao Li Zhenwei Shi
Domain invariant learning aims to learn models that extract invariant
features over various training domains, resulting in better generalization to
unseen target domains. Recently, Bayesian Neural Networks have achieved
promising results in domain invariant learning, but most works concentrate on
aligning features distributions rather than parameter distributions. Inspired
by the principle of Bayesian Neural Network, we attempt to directly learn the
domain invariant posterior distribution of network parameters. We first propose
a theorem to show that the invariant posterior of parameters can be implicitly
inferred by aggregating posteriors on different training domains. Our
assumption is more relaxed and allows us to extract more domain invariant
information. We also propose a simple yet effective method, named PosTerior
Generalization (PTG), that can be used to estimate the invariant parameter
distribution. PTG fully exploits variational inference to approximate parameter
distributions, including the invariant posterior and the posteriors on training
domains. Furthermore, we develop a lite version of PTG for widespread
applications. PTG shows competitive performance on various domain
generalization benchmarks on DomainBed. Additionally, PTG can use any existing
domain generalization methods as its prior, and combined with previous
state-of-the-art method the performance can be further improved. Code will be
made public.