Towards Detecting Contextual Real-Time Toxicity for In-Game Chat
Zachary Yang Nicolas Grenan-Godbout Reihaneh Rabbany
Real-time toxicity detection in online environments poses a significant
challenge, due to the increasing prevalence of social media and gaming
platforms. We introduce ToxBuster, a simple and scalable model that reliably
detects toxic content in real-time for a line of chat by including chat history
and metadata. ToxBuster consistently outperforms conventional toxicity models
across popular multiplayer games, including Rainbow Six Siege, For Honor, and
DOTA 2. We conduct an ablation study to assess the importance of each model
component and explore ToxBuster's transferability across the datasets.
Furthermore, we showcase ToxBuster's efficacy in post-game moderation,
successfully flagging 82.1% of chat-reported players at a precision level of
90.0%. Additionally, we show how an additional 6% of unreported toxic players
can be proactively moderated.