Domain Generalization Guided by Gradient Signal to Noise Ratio of
  Parameters
Mateusz Michalkiewicz Masoud Faraki Xiang Yu Manmohan Chandraker Mahsa Baktashmotlagh
Overfitting to the source domain is a common issue in gradient-based training
of deep neural networks. To compensate for the over-parameterized models,
numerous regularization techniques have been introduced such as those based on
dropout. While these methods achieve significant improvements on classical
benchmarks such as ImageNet, their performance diminishes with the introduction
of domain shift in the test set i.e. when the unseen data comes from a
significantly different distribution. In this paper, we move away from the
classical approach of Bernoulli sampled dropout mask construction and propose
to base the selection on gradient-signal-to-noise ratio (GSNR) of network's
parameters. Specifically, at each training step, parameters with high GSNR will
be discarded. Furthermore, we alleviate the burden of manually searching for
the optimal dropout ratio by leveraging a meta-learning approach. We evaluate
our method on standard domain generalization benchmarks and achieve competitive
results on classification and face anti-spoofing problems.