FTBC: Forward Temporal Bias Correction for Optimizing ANN-SNN Conversion
Xiaofeng Wu Velibor Bojkovic Bin Gu Kun Suo Kai Zou
Spiking Neural Networks (SNNs) offer a promising avenue for energy-efficient
computing compared with Artificial Neural Networks (ANNs), closely mirroring
biological neural processes. However, this potential comes with inherent
challenges in directly training SNNs through spatio-temporal backpropagation --
stemming from the temporal dynamics of spiking neurons and their discrete
signal processing -- which necessitates alternative ways of training, most
notably through ANN-SNN conversion. In this work, we introduce a lightweight
Forward Temporal Bias Correction (FTBC) technique, aimed at enhancing
conversion accuracy without the computational overhead. We ground our method on
provided theoretical findings that through proper temporal bias calibration the
expected error of ANN-SNN conversion can be reduced to be zero after each time
step. We further propose a heuristic algorithm for finding the temporal bias
only in the forward pass, thus eliminating the computational burden of
backpropagation and we evaluate our method on CIFAR-10/100 and ImageNet
datasets, achieving a notable increase in accuracy on all datasets. Codes are
released at a GitHub repository.