A Fast Algorithm to Simulate Nonlinear Resistive Networks
Benjamin Scellier
In the quest for energy-efficient artificial intelligence systems, resistor
networks are attracting interest as an alternative to conventional GPU-based
neural networks. These networks leverage the physics of electrical circuits for
inference and can be optimized with local training techniques such as
equilibrium propagation. Despite their potential advantage in terms of power
consumption, the challenge of efficiently simulating these resistor networks
has been a significant bottleneck to assess their scalability, with current
methods either being limited to linear networks or relying on realistic, yet
slow circuit simulators like SPICE. Assuming ideal circuit elements, we
introduce a novel approach for the simulation of nonlinear resistive networks,
which we frame as a quadratic programming problem with linear inequality
constraints, and which we solve using a fast, exact coordinate descent
algorithm. Our simulation methodology significantly outperforms existing
SPICE-based simulations, enabling the training of networks up to 325 times
larger at speeds 150 times faster, resulting in a 50,000-fold improvement in
the ratio of network size to epoch duration. Our approach, adaptable to other
electrical components, can foster more rapid progress in the simulations of
nonlinear electrical networks.