Conditional Logical Message Passing Transformer for Complex Query
  Answering
Chongzhi Zhang Zhiping Peng Junhao Zheng Qianli Ma
Complex Query Answering (CQA) over Knowledge Graphs (KGs) is a challenging
task. Given that KGs are usually incomplete, neural models are proposed to
solve CQA by performing multi-hop logical reasoning. However, most of them
cannot perform well on both one-hop and multi-hop queries simultaneously.
Recent work proposes a logical message passing mechanism based on the
pre-trained neural link predictors. While effective on both one-hop and
multi-hop queries, it ignores the difference between the constant and variable
nodes in a query graph. In addition, during the node embedding update stage,
this mechanism cannot dynamically measure the importance of different messages,
and whether it can capture the implicit logical dependencies related to a node
and received messages remains unclear. In this paper, we propose Conditional
Logical Message Passing Transformer (CLMPT), which considers the difference
between constants and variables in the case of using pre-trained neural link
predictors and performs message passing conditionally on the node type. We
empirically verified that this approach can reduce computational costs without
affecting performance. Furthermore, CLMPT uses the transformer to aggregate
received messages and update the corresponding node embedding. Through the
self-attention mechanism, CLMPT can assign adaptive weights to elements in an
input set consisting of received messages and the corresponding node and
explicitly model logical dependencies between various elements. Experimental
results show that CLMPT is a new state-of-the-art neural CQA model.