HiBid: A Cross-Channel Constrained Bidding System with Budget Allocation
  by Hierarchical Offline Deep Reinforcement Learning
Hao Wang Bo Tang Chi Harold Liu Shangqin Mao Jiahong Zhou Zipeng Dai Yaqi Sun Qianlong Xie Xingxing Wang Dong Wang
Online display advertising platforms service numerous advertisers by
providing real-time bidding (RTB) for the scale of billions of ad requests
every day. The bidding strategy handles ad requests cross multiple channels to
maximize the number of clicks under the set financial constraints, i.e., total
budget and cost-per-click (CPC), etc. Different from existing works mainly
focusing on single channel bidding, we explicitly consider cross-channel
constrained bidding with budget allocation. Specifically, we propose a
hierarchical offline deep reinforcement learning (DRL) framework called
``HiBid'', consisted of a high-level planner equipped with auxiliary loss for
non-competitive budget allocation, and a data augmentation enhanced low-level
executor for adaptive bidding strategy in response to allocated budgets.
Additionally, a CPC-guided action selection mechanism is introduced to satisfy
the cross-channel CPC constraint. Through extensive experiments on both the
large-scale log data and online A/B testing, we confirm that HiBid outperforms
six baselines in terms of the number of clicks, CPC satisfactory ratio, and
return-on-investment (ROI). We also deploy HiBid on Meituan advertising
platform to already service tens of thousands of advertisers every day.