SegMix: A Simple Structure-Aware Data Augmentation Method
Yuxin Pei Pushkar Bhuse Zhengzhong Liu Eric Xing
Interpolation-based Data Augmentation (DA) methods (Mixup) linearly
interpolate the inputs and labels of two or more training examples. Mixup has
more recently been adapted to the field of Natural Language Processing (NLP),
mainly for sequence labeling tasks. However, such a simple adoption yields
mixed or unstable improvements over the baseline models. We argue that the
direct-adoption methods do not account for structures in NLP tasks. To this
end, we propose SegMix, a collection of interpolation-based DA algorithms that
can adapt to task-specific structures. SegMix poses fewer constraints on data
structures, is robust to various hyperparameter settings, applies to more task
settings, and adds little computational overhead. In the algorithm's core, we
apply interpolation methods on task-specific meaningful segments, in contrast
to applying them on sequences as in prior work. We find SegMix to be a flexible
framework that combines rule-based DA methods with interpolation-based methods,
creating interesting mixtures of DA techniques. We show that SegMix
consistently improves performance over strong baseline models in Named Entity
Recognition (NER) and Relation Extraction (RE) tasks, especially under
data-scarce settings. Furthermore, this method is easy to implement and adds
negligible training overhead.