IR2: Information Regularization for Information Retrieval
Jianyou Wang Kaicheng Wang Xiaoyue Wang Weili Cao Ramamohan Paturi Leon Bergen
Effective information retrieval (IR) in settings with limited training data,
particularly for complex queries, remains a challenging task. This paper
introduces IR2, Information Regularization for Information Retrieval, a
technique for reducing overfitting during synthetic data generation. This
approach, representing a novel application of regularization techniques in
synthetic data creation for IR, is tested on three recent IR tasks
characterized by complex queries: DORIS-MAE, ArguAna, and WhatsThatBook.
Experimental results indicate that our regularization techniques not only
outperform previous synthetic query generation methods on the tasks considered
but also reduce cost by up to 50%. Furthermore, this paper categorizes and
explores three regularization methods at different stages of the query
synthesis pipeline-input, prompt, and output-each offering varying degrees of
performance improvement compared to models where no regularization is applied.
This provides a systematic approach for optimizing synthetic data generation in
data-limited, complex-query IR scenarios. All code, prompts and synthetic data
are available at
https://github.com/Info-Regularization/Information-Regularization.