Learning Differentiable Logic Programs for Abstract Visual Reasoning
Hikaru Shindo Viktor Pfanschilling Devendra Singh Dhami Kristian Kersting
Visual reasoning is essential for building intelligent agents that understand
the world and perform problem-solving beyond perception. Differentiable forward
reasoning has been developed to integrate reasoning with gradient-based machine
learning paradigms. However, due to the memory intensity, most existing
approaches do not bring the best of the expressivity of first-order logic,
excluding a crucial ability to solve abstract visual reasoning, where agents
need to perform reasoning by using analogies on abstract concepts in different
scenarios. To overcome this problem, we propose NEUro-symbolic Message-pAssiNg
reasoNer (NEUMANN), which is a graph-based differentiable forward reasoner,
passing messages in a memory-efficient manner and handling structured programs
with functors. Moreover, we propose a computationally-efficient structure
learning algorithm to perform explanatory program induction on complex visual
scenes. To evaluate, in addition to conventional visual reasoning tasks, we
propose a new task, visual reasoning behind-the-scenes, where agents need to
learn abstract programs and then answer queries by imagining scenes that are
not observed. We empirically demonstrate that NEUMANN solves visual reasoning
tasks efficiently, outperforming neural, symbolic, and neuro-symbolic
baselines.