Unsupervised hard Negative Augmentation for contrastive learning
Yuxuan Shu Vasileios Lampos
We present Unsupervised hard Negative Augmentation (UNA), a method that
generates synthetic negative instances based on the term frequency-inverse
document frequency (TF-IDF) retrieval model. UNA uses TF-IDF scores to
ascertain the perceived importance of terms in a sentence and then produces
negative samples by replacing terms with respect to that. Our experiments
demonstrate that models trained with UNA improve the overall performance in
semantic textual similarity tasks. Additional performance gains are obtained
when combining UNA with the paraphrasing augmentation. Further results show
that our method is compatible with different backbone models. Ablation studies
also support the choice of having a TF-IDF-driven control on negative
augmentation.