Despite impressive performance as evaluated on i.i.d. holdout data, deep neural networks depend heavily on superficial statistics of the training data and are liable to break under distribution shift. For example, subtle changes to the background or texture of an image can break a seemingly powerful classifier. Building on previous work on domain generalization, we hope to produce a classifier that will generalize to previously unseen domains, even when domain identifiers are not available during training. This setting is challenging because the model may extract many distribution-specific (superficial) signals together with distribution-agnostic (semantic) signals. To overcome this challenge, we incorporate the gray-level co-occurrence matrix (GLCM) to extract patterns that our prior knowledge suggests are superficial: they are sensitive to the texture but unable to capture the gestalt of an image. Then we introduce two techniques for improving our networks' out-of-sample performance. The first method is built on the reverse gradient method that pushes our model to learn representations from which the GLCM representation is not predictable. The second method is built on the independence introduced by projecting the model's representation onto the subspace orthogonal to GLCM representation's.
 We test our method on the battery of standard domain generalization data sets and, interestingly, achieve comparable or better performance as compared to other domain generalization methods that explicitly require samples from the target distribution for training. Imagine training an image classifier to recognize facial expressions. In the training data, while all images labeled "smile" may actually depict smiling people, the "smile" label might also be correlated with other aspects of the image. For example, people might tend to smile more often while outdoors, and to frown more in airports. In the future, we might encounter photographs with previously unseen backgrounds, and thus we prefer models that rely as little as possible on the superficial signal.The problem of learning classifiers robust to distribution shift, commonly called Domain Adaptation (DA), has a rich history. Under restrictive assumptions, such as covariate shift BID43 BID16 , and label shift (also known as target shift or prior probability shift) BID44 BID41 BID48 BID30 , principled methods exist for estimating the shifts and retraining under the importance-weighted ERM framework. Other papers bound worst-case performance under bounded shifts as measured by divergence measures on the train v.s. test distributions BID3 BID33 BID20 .While many impossibility results for DA have been proven BID4 , humans nevertheless exhibit a remarkable ability to function out-of-sample, even when confronting dramatic Example illustration of train/validation/test data. The first row is "happiness" sentiment and the second row is "sadness" sentiment. The background and sentiment labels are correlated in training and validation set, but independent in testing set.distribution shift. Few would doubt that given photographs of smiling and frowning astronauts on the Martian plains, we could (mostly) agree upon the correct labels.While we lack a mathematical description of how precisely humans are able to generalize so easily out-of-sample, we can often point to certain classes of perturbations that should not effect the semantics of an image. For example for many tasks, we know that the background should not influence the predictions made about an image. Similarly, other superficial statistics of the data, such as textures or subtle coloring changes should not matter. The essential assumption of this paper is that by making our model depend less on known superficial aspects, we can push the model to rely more on the difference that makes a difference. This paper focuses on visual applications, and we focus on high-frequency textural information as the relevant notion of superficial statistics that we do not want our model to depend upon.The contribution of this paper can be summarized as follows.• We propose a new differentiable neural network building block (neural gray-level cooccurrence matrix) that captures textural information only from images without modeling the lower-frequency semantic information that we care about (Section 3.1).• We propose an architecture-agnostic , parameter-free method that is designed to discard this superficial information, (Section 3.2).• We introduce two synthetic datasets for DA/DG studies that are more challenging than regular DA/DG scenario in the sense that the domain-specific information is correlated with semantic information. FIG0 is a toy example (Section 4). We introduced two novel components: NGLCM that only extracts textural information from an image, and HEX that projects the textural information out and forces the model to focus on semantic information. Limitations still exist. For example, NGLCM cannot be completely free of semantic information of an image. As a result, if we apply our method on standard MNIST data set, we will 3 ) learns garbage information and HEX degenerates to the baseline model. To overcome these limitations, we invented several training heuristics, such as optimizing F P and F G sequentially and then fix some weights. However, we did not report results with training heuristics (expect for PACS experiment) because we hope to simplify the methods. Another limitation we observe is that sometimes the training performance of HEX fluctuates dramatically during training, but fortunately, the model picked up by highest validation accuracy generally performs better than competing methods. Despite these limitations, we still achieved impressive performance on both synthetic and popular DG data sets.
Building on previous work on domain generalization, we hope to produce a classifier that will generalize to previously unseen domains, even when domain identifiers are not available during training. A domain generalization approach to reveal semantic information based on a linear projection scheme from CNN and NGLCM output layers. The paper proposes an unsupervised approach to identify image features that are not meaningful for image classification tasks