Sentiment-enhanced Graph-based Sarcasm Explanation in Dialogue
Kun Ouyang Liqiang Jing Xuemeng Song Meng Liu Yupeng Hu Liqiang Nie
Sarcasm Explanation in Dialogue (SED) is a new yet challenging task, which
aims to generate a natural language explanation for the given sarcastic
dialogue that involves multiple modalities (i.e., utterance, video, and audio).
Although existing studies have achieved great success based on the generative
pretrained language model BART, they overlook exploiting the sentiments
residing in the utterance, video and audio, which are vital clues for sarcasm
explanation. In fact, it is non-trivial to incorporate sentiments for boosting
SED performance, due to three main challenges: 1) diverse effects of utterance
tokens on sentiments; 2) gap between video-audio sentiment signals and the
embedding space of BART; and 3) various relations among utterances, utterance
sentiments, and video-audio sentiments. To tackle these challenges, we propose
a novel sEntiment-enhanceD Graph-based multimodal sarcasm Explanation
framework, named EDGE. In particular, we first propose a lexicon-guided
utterance sentiment inference module, where a heuristic utterance sentiment
refinement strategy is devised. We then develop a module named Joint Cross
Attention-based Sentiment Inference (JCA-SI) by extending the multimodal
sentiment analysis model JCA to derive the joint sentiment label for each
video-audio clip. Thereafter, we devise a context-sentiment graph to
comprehensively model the semantic relations among the utterances, utterance
sentiments, and video-audio sentiments, to facilitate sarcasm explanation
generation. Extensive experiments on the publicly released dataset WITS verify
the superiority of our model over cutting-edge methods.