Comparing Differentiable Logics for Learning Systems: A Research Preview
Thomas Flinkow Barak A. Pearlmutter Rosemary Monahan
Extensive research on formal verification of machine learning (ML) systems
indicates that learning from data alone often fails to capture underlying
background knowledge. A variety of verifiers have been developed to ensure that
a machine-learnt model satisfies correctness and safety properties, however,
these verifiers typically assume a trained network with fixed weights.
ML-enabled autonomous systems are required to not only detect incorrect
predictions, but should also possess the ability to self-correct, continuously
improving and adapting. A promising approach for creating ML models that
inherently satisfy constraints is to encode background knowledge as logical
constraints that guide the learning process via so-called differentiable
logics. In this research preview, we compare and evaluate various logics from
the literature in weakly-supervised contexts, presenting our findings and
highlighting open problems for future work. Our experimental results are
broadly consistent with results reported previously in literature; however,
learning with differentiable logics introduces a new hyperparameter that is
difficult to tune and has significant influence on the effectiveness of the
logics.