ContentCTR: Frame-level Live Streaming Click-Through Rate Prediction
  with Multimodal Transformer
Jiaxin Deng Dong Shen Shiyao Wang Xiangyu Wu Fan Yang Guorui Zhou Gaofeng Meng
In recent years, live streaming platforms have gained immense popularity as
they allow users to broadcast their videos and interact in real-time with hosts
and peers. Due to the dynamic changes of live content, accurate recommendation
models are crucial for enhancing user experience. However, most previous works
treat the live as a whole item and explore the Click-through-Rate (CTR)
prediction framework on item-level, neglecting that the dynamic changes that
occur even within the same live room. In this paper, we proposed a ContentCTR
model that leverages multimodal transformer for frame-level CTR prediction.
First, we present an end-to-end framework that can make full use of multimodal
information, including visual frames, audio, and comments, to identify the most
attractive live frames. Second, to prevent the model from collapsing into a
mediocre solution, a novel pairwise loss function with first-order difference
constraints is proposed to utilize the contrastive information existing in the
highlight and non-highlight frames. Additionally, we design a temporal
text-video alignment module based on Dynamic Time Warping to eliminate noise
caused by the ambiguity and non-sequential alignment of visual and textual
information. We conduct extensive experiments on both real-world scenarios and
public datasets, and our ContentCTR model outperforms traditional
recommendation models in capturing real-time content changes. Moreover, we
deploy the proposed method on our company platform, and the results of online
A/B testing further validate its practical significance.