An IPW-based Unbiased Ranking Metric in Two-sided Markets
Keisho Oh Naoki Nishimura Minje Sung Ken Kobayashi Kazuhide Nakata
In modern recommendation systems, unbiased learning-to-rank (LTR) is crucial
for prioritizing items from biased implicit user feedback, such as click data.
Several techniques, such as Inverse Propensity Weighting (IPW), have been
proposed for single-sided markets. However, less attention has been paid to
two-sided markets, such as job platforms or dating services, where successful
conversions require matching preferences from both users. This paper addresses
the complex interaction of biases between users in two-sided markets and
proposes a tailored LTR approach. We first present a formulation of feedback
mechanisms in two-sided matching platforms and point out that their implicit
feedback may include position bias from both user groups. On the basis of this
observation, we extend the IPW estimator and propose a new estimator, named
two-sided IPW, to address the position bases in two-sided markets. We prove
that the proposed estimator satisfies the unbiasedness for the ground-truth
ranking metric. We conducted numerical experiments on real-world two-sided
platforms and demonstrated the effectiveness of our proposed method in terms of
both precision and robustness. Our experiments showed that our method
outperformed baselines especially when handling rare items, which are less
frequently observed in the training data.