Training Multi-layer Neural Networks on Ising Machine
Xujie Song Tong Liu Shengbo Eben Li Jingliang Duan Wenxuan Wang Keqiang Li
As a dedicated quantum device, Ising machines could solve large-scale binary
optimization problems in milliseconds. There is emerging interest in utilizing
Ising machines to train feedforward neural networks due to the prosperity of
generative artificial intelligence. However, existing methods can only train
single-layer feedforward networks because of the complex nonlinear network
topology. This paper proposes an Ising learning algorithm to train quantized
neural network (QNN), by incorporating two essential techinques, namely binary
representation of topological network and order reduction of loss function. As
far as we know, this is the first algorithm to train multi-layer feedforward
networks on Ising machines, providing an alternative to gradient-based
backpropagation. Firstly, training QNN is formulated as a quadratic constrained
binary optimization (QCBO) problem by representing neuron connection and
activation function as equality constraints. All quantized variables are
encoded by binary bits based on binary encoding protocol. Secondly, QCBO is
converted to a quadratic unconstrained binary optimization (QUBO) problem, that
can be efficiently solved on Ising machines. The conversion leverages both
penalty function and Rosenberg order reduction, who together eliminate equality
constraints and reduce high-order loss function into a quadratic one. With some
assumptions, theoretical analysis shows the space complexity of our algorithm
is $\mathcal{O}(H^2L + HLN\log H)$, quantifying the required number of Ising
spins. Finally, the algorithm effectiveness is validated with a simulated Ising
machine on MNIST dataset. After annealing 700 ms, the classification accuracy
achieves 98.3%. Among 100 runs, the success probability of finding the optimal
solution is 72%. Along with the increasing number of spins on Ising machine,
our algorithm has the potential to train deeper neural networks.