Semantic Equivariant Mixup
Zongbo Han Tianchi Xie Bingzhe Wu Qinghua Hu Changqing Zhang
Mixup is a well-established data augmentation technique, which can extend the
training distribution and regularize the neural networks by creating ''mixed''
samples based on the label-equivariance assumption, i.e., a proportional mixup
of the input data results in the corresponding labels being mixed in the same
proportion. However, previous mixup variants may fail to exploit the
label-independent information in mixed samples during training, which usually
contains richer semantic information. To further release the power of mixup, we
first improve the previous label-equivariance assumption by the
semantic-equivariance assumption, which states that the proportional mixup of
the input data should lead to the corresponding representation being mixed in
the same proportion. Then a generic mixup regularization at the representation
level is proposed, which can further regularize the model with the semantic
information in mixed samples. At a high level, the proposed semantic
equivariant mixup (sem) encourages the structure of the input data to be
preserved in the representation space, i.e., the change of input will result in
the obtained representation information changing in the same way. Different
from previous mixup variants, which tend to over-focus on the label-related
information, the proposed method aims to preserve richer semantic information
in the input with semantic-equivariance assumption, thereby improving the
robustness of the model against distribution shifts. We conduct extensive
empirical studies and qualitative analyzes to demonstrate the effectiveness of
our proposed method. The code of the manuscript is in the supplement.