Confidence-Aware Multi-Field Model Calibration
Yuang Zhao Chuhan Wu Qinglin Jia Hong Zhu Jia Yan Libin Zong Linxuan Zhang Zhenhua Dong Muyu Zhang
Accurately predicting the probabilities of user feedback, such as clicks and
conversions, is critical for ad ranking and bidding. However, there often exist
unwanted mismatches between predicted probabilities and true likelihoods due to
the shift of data distributions and intrinsic model biases. Calibration aims to
address this issue by post-processing model predictions, and field-aware
calibration can adjust model output on different feature field values to
satisfy fine-grained advertising demands. Unfortunately, the observed samples
corresponding to certain field values can be too limited to make confident
calibrations, which may yield bias amplification and online disturbance. In
this paper, we propose a confidence-aware multi-field calibration method, which
adaptively adjusts the calibration intensity based on the confidence levels
derived from sample statistics. It also utilizes multiple feature fields for
joint model calibration with awareness of their importance to mitigate the data
sparsity effect of a single field. Extensive offline and online experiments
show the superiority of our method in boosting advertising performance and
reducing prediction deviations.