Ad Recommendation in a Collapsed and Entangled World
Junwei Pan Wei Xue Ximei Wang Haibin Yu Xun Liu Shijie Quan Xueming Qiu Dapeng Liu Lei Xiao Jie Jiang
In this paper, we present an industry ad recommendation system, paying
attention to the challenges and practices of learning appropriate
representations. Our study begins by showcasing our approaches to preserving
priors when encoding features of diverse types into embedding representations.
Specifically, we address sequence features, numeric features, pre-trained
embedding features, as well as sparse ID features. Moreover, we delve into two
pivotal challenges associated with feature representation: the dimensional
collapse of embeddings and the interest entanglement across various tasks or
scenarios. Subsequently, we propose several practical approaches to effectively
tackle these two challenges. We then explore several training techniques to
facilitate model optimization, reduce bias, and enhance exploration.
Furthermore, we introduce three analysis tools that enable us to
comprehensively study feature correlation, dimensional collapse, and interest
entanglement. This work builds upon the continuous efforts of Tencent's ads
recommendation team in the last decade. It not only summarizes general design
principles but also presents a series of off-the-shelf solutions and analysis
tools. The reported performance is based on our online advertising platform,
which handles hundreds of billions of requests daily, serving millions of ads
to billions of users.