Counterfactual Reasoning with Knowledge Graph Embeddings
Lena Zellinger Andreas Stephan Benjamin Roth
Knowledge graph embeddings (KGEs) were originally developed to infer true but
missing facts in incomplete knowledge repositories. In this paper, we link
knowledge graph completion and counterfactual reasoning via our new task CFKGR.
We model the original world state as a knowledge graph, hypothetical scenarios
as edges added to the graph, and plausible changes to the graph as inferences
from logical rules. We create corresponding benchmark datasets, which contain
diverse hypothetical scenarios with plausible changes to the original knowledge
graph and facts that should be retained. We develop COULDD, a general method
for adapting existing knowledge graph embeddings given a hypothetical premise,
and evaluate it on our benchmark. Our results indicate that KGEs learn patterns
in the graph without explicit training. We further observe that KGEs adapted
with COULDD solidly detect plausible counterfactual changes to the graph that
follow these patterns. An evaluation on human-annotated data reveals that KGEs
adapted with COULDD are mostly unable to recognize changes to the graph that do
not follow learned inference rules. In contrast, ChatGPT mostly outperforms
KGEs in detecting plausible changes to the graph but has poor knowledge
retention. In summary, CFKGR connects two previously distinct areas, namely KG
completion and counterfactual reasoning.