Optimisation in Neurosymbolic Learning Systems
Emile van Krieken
Neurosymbolic AI aims to integrate deep learning with symbolic AI. This
integration has many promises, such as decreasing the amount of data required
to train a neural network, improving the explainability and interpretability of
answers given by models and verifying the correctness of trained systems. We
study neurosymbolic learning, where we have both data and background knowledge
expressed using symbolic languages. How do we connect the symbolic and neural
components to communicate this knowledge? One option is fuzzy reasoning, which
studies degrees of truth. For example, being tall is not a binary concept.
Instead, probabilistic reasoning studies the probability that something is true
or will happen. Our first research question studies how different forms of
fuzzy reasoning combine with learning. We find surprising results like a
connection to the Raven paradox stating we confirm "ravens are black" when we
observe a green apple. In this study, we did not use the background knowledge
when we deployed our models after training. In our second research question, we
studied how to use background knowledge in deployed models. We developed a new
neural network layer based on fuzzy reasoning. Probabilistic reasoning is a
natural fit for neural networks, which we usually train to be probabilistic.
However, they are expensive to compute and do not scale well to large tasks. In
our third research question, we study how to connect probabilistic reasoning
with neural networks by sampling to estimate averages, while in the final
research question, we study scaling probabilistic neurosymbolic learning to
much larger problems than before. Our insight is to train a neural network with
synthetic data to predict the result of probabilistic reasoning.