SSP: Self-Supervised Post-training for Conversational Search
Quan Tu Shen Gao Xiaolong Wu Zhao Cao Ji-Rong Wen Rui Yan
Conversational search has been regarded as the next-generation search
paradigm. Constrained by data scarcity, most existing methods distill the
well-trained ad-hoc retriever to the conversational retriever. However, these
methods, which usually initialize parameters by query reformulation to discover
contextualized dependency, have trouble in understanding the dialogue structure
information and struggle with contextual semantic vanishing. In this paper, we
propose \fullmodel (\model) which is a new post-training paradigm with three
self-supervised tasks to efficiently initialize the conversational search model
to enhance the dialogue structure and contextual semantic understanding.
Furthermore, the \model can be plugged into most of the existing conversational
models to boost their performance. To verify the effectiveness of our proposed
method, we apply the conversational encoder post-trained by \model on the
conversational search task using two benchmark datasets: CAsT-19 and CAsT-20.
Extensive experiments that our \model can boost the performance of several
existing conversational search methods. Our source code is available at
\url{https://github.com/morecry/SSP}.