Improving Sentence Embeddings with an Automatically Generated NLI
  Dataset
Soma Sato Hayato Tsukagoshi Ryohei Sasano Koichi Takeda
Decoder-based large language models (LLMs) have shown high performance on
many tasks in natural language processing. This is also true for sentence
embedding learning, where a decoder-based model, PromptEOL, has achieved the
best performance on semantic textual similarity (STS) tasks. However, PromptEOL
makes great use of fine-tuning with a manually annotated natural language
inference (NLI) dataset. We aim to improve sentence embeddings learned in an
unsupervised setting by automatically generating an NLI dataset with an LLM and
using it to fine-tune PromptEOL. In experiments on STS tasks, the proposed
method achieved an average Spearman's rank correlation coefficient of 82.21
with respect to human evaluation, thus outperforming existing methods without
using large, manually annotated datasets.