Probabilistic Modeling: Proving the Lottery Ticket Hypothesis in Spiking
  Neural Network
Man Yao Yuhong Chou Guangshe Zhao Xiawu Zheng Yonghong Tian Bo Xu Guoqi Li
The Lottery Ticket Hypothesis (LTH) states that a randomly-initialized large
neural network contains a small sub-network (i.e., winning tickets) which, when
trained in isolation, can achieve comparable performance to the large network.
LTH opens up a new path for network pruning. Existing proofs of LTH in
Artificial Neural Networks (ANNs) are based on continuous activation functions,
such as ReLU, which satisfying the Lipschitz condition. However, these
theoretical methods are not applicable in Spiking Neural Networks (SNNs) due to
the discontinuous of spiking function. We argue that it is possible to extend
the scope of LTH by eliminating Lipschitz condition. Specifically, we propose a
novel probabilistic modeling approach for spiking neurons with complicated
spatio-temporal dynamics. Then we theoretically and experimentally prove that
LTH holds in SNNs. According to our theorem, we conclude that pruning directly
in accordance with the weight size in existing SNNs is clearly not optimal. We
further design a new criterion for pruning based on our theory, which achieves
better pruning results than baseline.