DRIN: Dynamic Relation Interactive Network for Multimodal Entity Linking
Shangyu Xing Fei Zhao Zhen Wu Chunhui Li Jianbing Zhang Xinyu Dai
Multimodal Entity Linking (MEL) is a task that aims to link ambiguous
mentions within multimodal contexts to referential entities in a multimodal
knowledge base. Recent methods for MEL adopt a common framework: they first
interact and fuse the text and image to obtain representations of the mention
and entity respectively, and then compute the similarity between them to
predict the correct entity. However, these methods still suffer from two
limitations: first, as they fuse the features of text and image before
matching, they cannot fully exploit the fine-grained alignment relations
between the mention and entity. Second, their alignment is static, leading to
low performance when dealing with complex and diverse data. To address these
issues, we propose a novel framework called Dynamic Relation Interactive
Network (DRIN) for MEL tasks. DRIN explicitly models four different types of
alignment between a mention and entity and builds a dynamic Graph Convolutional
Network (GCN) to dynamically select the corresponding alignment relations for
different input samples. Experiments on two datasets show that DRIN outperforms
state-of-the-art methods by a large margin, demonstrating the effectiveness of
our approach.