Improving Topic Relevance Model by Mix-structured Summarization and
  LLM-based Data Augmentation
Yizhu Liu Ran Tao Shengyu Guo Yifan Yang
Topic relevance between query and document is a very important part of social
search, which can evaluate the degree of matching between document and user's
requirement. In most social search scenarios such as Dianping, modeling search
relevance always faces two challenges. One is that many documents in social
search are very long and have much redundant information. The other is that the
training data for search relevance model is difficult to get, especially for
multi-classification relevance model. To tackle above two problems, we first
take query concatenated with the query-based summary and the document summary
without query as the input of topic relevance model, which can help model learn
the relevance degree between query and the core topic of document. Then, we
utilize the language understanding and generation abilities of large language
model (LLM) to rewrite and generate query from queries and documents in
existing training data, which can construct new query-document pairs as
training data. Extensive offline experiments and online A/B tests show that the
proposed approaches effectively improve the performance of relevance modeling.