Sparse-firing regularization methods for spiking neural networks with
  time-to-first spike coding
Yusuke Sakemi Kakei Yamamoto Takeo Hosomi Kazuyuki Aihara
The training of multilayer spiking neural networks (SNNs) using the error
backpropagation algorithm has made significant progress in recent years. Among
the various training schemes, the error backpropagation method that directly
uses the firing time of neurons has attracted considerable attention because it
can realize ideal temporal coding. This method uses time-to-first spike (TTFS)
coding, in which each neuron fires at most once, and this restriction on the
number of firings enables information to be processed at a very low firing
frequency. This low firing frequency increases the energy efficiency of
information processing in SNNs, which is important not only because of its
similarity with information processing in the brain, but also from an
engineering point of view. However, only an upper limit has been provided for
TTFS-coded SNNs, and the information-processing capability of SNNs at lower
firing frequencies has not been fully investigated. In this paper, we propose
two spike timing-based sparse-firing (SSR) regularization methods to further
reduce the firing frequency of TTFS-coded SNNs. The first is the membrane
potential-aware SSR (M-SSR) method, which has been derived as an extreme form
of the loss function of the membrane potential value. The second is the firing
condition-aware SSR (F-SSR) method, which is a regularization function obtained
from the firing conditions. Both methods are characterized by the fact that
they only require information about the firing timing and associated weights.
The effects of these regularization methods were investigated on the MNIST,
Fashion-MNIST, and CIFAR-10 datasets using multilayer perceptron networks and
convolutional neural network structures.