Fast gradient-free activation maximization for neurons in spiking neural
  networks
Nikita Pospelov Andrei Chertkov Maxim Beketov Ivan Oseledets Konstantin Anokhin
Neural networks (NNs), both living and artificial, work due to being complex
systems of neurons, each having its own specialization. Revealing these
specializations is important for understanding NNs inner working mechanisms.
The only way to do this for a living system, the neural response of which to a
stimulus is not a known (let alone differentiable) function is to build a
feedback loop of exposing it to stimuli, the properties of which can be
iteratively varied aiming in the direction of maximal response. To test such a
loop on a living network, one should first learn how to run it quickly and
efficiently, reaching most effective stimuli (ones that maximize certain
neurons activation) in least possible number of iterations. We present a
framework with an effective design of such a loop, successfully testing it on
an artificial spiking neural network (SNN, a model that mimics the behaviour of
NNs in living brains). Our optimization method used for activation maximization
(AM) was based on low-rank tensor decomposition (Tensor Train, TT) of the
activation function's discretization over its domain the latent parameter space
of stimuli (CIFAR10-size color images, generated by either VQ-VAE or SN-GAN
from their latent description vectors, fed to the SNN). To our knowledge, the
present work is the first attempt to perform effective AM for SNNs. The source
code of our framework, MANGO (for Maximization of neural Activation via
Non-Gradient Optimization) is available on GitHub.