An Automata-Theoretic Approach to Synthesizing Binarized Neural Networks
Ye Tao Wanwei Liu Fu Song Zhen Liang Ji Wang Hongxu Zhu
Deep neural networks, (DNNs, a.k.a. NNs), have been widely used in various
tasks and have been proven to be successful. However, the accompanied expensive
computing and storage costs make the deployments in resource-constrained
devices a significant concern. To solve this issue, quantization has emerged as
an effective way to reduce the costs of DNNs with little accuracy degradation
by quantizing floating-point numbers to low-width fixed-point representations.
Quantized neural networks (QNNs) have been developed, with binarized neural
networks (BNNs) restricted to binary values as a special case. Another concern
about neural networks is their vulnerability and lack of interpretability.
Despite the active research on trustworthy of DNNs, few approaches have been
proposed to QNNs. To this end, this paper presents an automata-theoretic
approach to synthesizing BNNs that meet designated properties. More
specifically, we define a temporal logic, called BLTL, as the specification
language. We show that each BLTL formula can be transformed into an automaton
on finite words. To deal with the state-explosion problem, we provide a
tableau-based approach in real implementation. For the synthesis procedure, we
utilize SMT solvers to detect the existence of a model (i.e., a BNN) in the
construction process. Notably, synthesis provides a way to determine the
hyper-parameters of the network before training.Moreover, we experimentally
evaluate our approach and demonstrate its effectiveness in improving the
individual fairness and local robustness of BNNs while maintaining accuracy to
a great extent.