INSURE: An Information Theory Inspired Disentanglement and Purification
  Model for Domain Generalization
Xi Yu Huan-Hsin Tseng Shinjae Yoo Haibin Ling Yuewei Lin
Domain Generalization (DG) aims to learn a generalizable model on the unseen
target domain by only training on the multiple observed source domains.
Although a variety of DG methods have focused on extracting domain-invariant
features, the domain-specific class-relevant features have attracted attention
and been argued to benefit generalization to the unseen target domain. To take
into account the class-relevant domain-specific information, in this paper we
propose an Information theory iNspired diSentanglement and pURification modEl
(INSURE) to explicitly disentangle the latent features to obtain sufficient and
compact (necessary) class-relevant feature for generalization to the unseen
domain. Specifically, we first propose an information theory inspired loss
function to ensure the disentangled class-relevant features contain sufficient
class label information and the other disentangled auxiliary feature has
sufficient domain information. We further propose a paired purification loss
function to let the auxiliary feature discard all the class-relevant
information and thus the class-relevant feature will contain sufficient and
compact (necessary) class-relevant information. Moreover, instead of using
multiple encoders, we propose to use a learnable binary mask as our
disentangler to make the disentanglement more efficient and make the
disentangled features complementary to each other. We conduct extensive
experiments on four widely used DG benchmark datasets including PACS,
OfficeHome, TerraIncognita, and DomainNet. The proposed INSURE outperforms the
state-of-art methods. We also empirically show that domain-specific
class-relevant features are beneficial for domain generalization.