SenTest: Evaluating Robustness of Sentence Encoders
Tanmay Chavan Shantanu Patankar Aditya Kane Omkar Gokhale Geetanjali Kale Raviraj Joshi
Contrastive learning has proven to be an effective method for pre-training
models using weakly labeled data in the vision domain. Sentence transformers
are the NLP counterparts to this architecture, and have been growing in
popularity due to their rich and effective sentence representations. Having
effective sentence representations is paramount in multiple tasks, such as
information retrieval, retrieval augmented generation (RAG), and sentence
comparison. Keeping in mind the deployability factor of transformers,
evaluating the robustness of sentence transformers is of utmost importance.
This work focuses on evaluating the robustness of the sentence encoders. We
employ several adversarial attacks to evaluate its robustness. This system uses
character-level attacks in the form of random character substitution,
word-level attacks in the form of synonym replacement, and sentence-level
attacks in the form of intra-sentence word order shuffling. The results of the
experiments strongly undermine the robustness of sentence encoders. The models
produce significantly different predictions as well as embeddings on perturbed
datasets. The accuracy of the models can fall up to 15 percent on perturbed
datasets as compared to unperturbed datasets. Furthermore, the experiments
demonstrate that these embeddings does capture the semantic and syntactic
structure (sentence order) of sentences. However, existing supervised
classification strategies fail to leverage this information, and merely
function as n-gram detectors.