Unbiased Filtering Of Accidental Clicks in Verizon Media Native
  Advertising
Yohay Kaplan Naama Krasne Alex Shtoff Oren Somekh
Verizon Media (VZM) native advertising is one of VZM largest and fastest
growing businesses, reaching a run-rate of several hundred million USDs in the
past year. Driving the VZM native models that are used to predict event
probabilities, such as click and conversion probabilities, is OFFSET - a
feature enhanced collaborative-filtering based event-prediction algorithm. In
this work we focus on the challenge of predicting click-through rates (CTR)
when we are aware that some of the clicks have short dwell-time and are defined
as accidental clicks. An accidental click implies little affinity between the
user and the ad, so predicting that similar users will click on the ad is
inaccurate. Therefore, it may be beneficial to remove clicks with dwell-time
lower than a predefined threshold from the training set. However, we cannot
ignore these positive events, as filtering these will cause the model to under
predict. Previous approaches have tried to apply filtering and then adding
corrective biases to the CTR predictions, but did not yield revenue lifts and
therefore were not adopted. In this work, we present a new approach where the
positive weight of the accidental clicks is distributed among all of the
negative events (skips), based on their likelihood of causing accidental
clicks, as predicted by an auxiliary model. These likelihoods are taken as the
correct labels of the negative events, shifting our training from using only
binary labels and adopting a binary cross-entropy loss function in our training
process. After showing offline performance improvements, the modified model was
tested online serving VZM native users, and provided 1.18% revenue lift over
the production model which is agnostic to accidental clicks.