On the Fly Neural Style Smoothing for Risk-Averse Domain Generalization
Akshay Mehra Yunbei Zhang Bhavya Kailkhura Jihun Hamm
Achieving high accuracy on data from domains unseen during training is a
fundamental challenge in domain generalization (DG). While state-of-the-art DG
classifiers have demonstrated impressive performance across various tasks, they
have shown a bias towards domain-dependent information, such as image styles,
rather than domain-invariant information, such as image content. This bias
renders them unreliable for deployment in risk-sensitive scenarios such as
autonomous driving where a misclassification could lead to catastrophic
consequences. To enable risk-averse predictions from a DG classifier, we
propose a novel inference procedure, Test-Time Neural Style Smoothing (TT-NSS),
that uses a "style-smoothed" version of the DG classifier for prediction at
test time. Specifically, the style-smoothed classifier classifies a test image
as the most probable class predicted by the DG classifier on random
re-stylizations of the test image. TT-NSS uses a neural style transfer module
to stylize a test image on the fly, requires only black-box access to the DG
classifier, and crucially, abstains when predictions of the DG classifier on
the stylized test images lack consensus. Additionally, we propose a neural
style smoothing (NSS) based training procedure that can be seamlessly
integrated with existing DG methods. This procedure enhances prediction
consistency, improving the performance of TT-NSS on non-abstained samples. Our
empirical results demonstrate the effectiveness of TT-NSS and NSS at producing
and improving risk-averse predictions on unseen domains from DG classifiers
trained with SOTA training methods on various benchmark datasets and their
variations.