Convex and Bilevel Optimization for Neuro-Symbolic Inference and
  Learning
Charles Dickens Changyu Gao Connor Pryor Stephen Wright Lise Getoor
We address a key challenge for neuro-symbolic (NeSy) systems by leveraging
convex and bilevel optimization techniques to develop a general gradient-based
framework for end-to-end neural and symbolic parameter learning. The
applicability of our framework is demonstrated with NeuPSL, a state-of-the-art
NeSy architecture. To achieve this, we propose a smooth primal and dual
formulation of NeuPSL inference and show learning gradients are functions of
the optimal dual variables. Additionally, we develop a dual block coordinate
descent algorithm for the new formulation that naturally exploits warm-starts.
This leads to over 100x learning runtime improvements over the current best
NeuPSL inference method. Finally, we provide extensive empirical evaluations
across $8$ datasets covering a range of tasks and demonstrate our learning
framework achieves up to a 16% point prediction performance improvement over
alternative learning methods.