Hybrid moderation in the newsroom: Recommending featured posts to
  content moderators
Cedric Waterschoot Antal van den Bosch
Online news outlets are grappling with the moderation of user-generated
content within their comment section. We present a recommender system based on
ranking class probabilities to support and empower the moderator in choosing
featured posts, a time-consuming task. By combining user and textual content
features we obtain an optimal classification F1-score of 0.44 on the test set.
Furthermore, we observe an optimum mean NDCG@5 of 0.87 on a large set of
validation articles. As an expert evaluation, content moderators assessed the
output of a random selection of articles by choosing comments to feature based
on the recommendations, which resulted in a NDCG score of 0.83. We conclude
that first, adding text features yields the best score and second, while
choosing featured content remains somewhat subjective, content moderators found
suitable comments in all but one evaluated recommendations. We end the paper by
analyzing our best-performing model, a step towards transparency and
explainability in hybrid content moderation.