Prime and Modulate Learning: Generation of forward models with signed
  back-propagation and environmental cues
Sama Daryanavard Bernd Porr
Deep neural networks employing error back-propagation for learning can suffer
from exploding and vanishing gradient problems. Numerous solutions have been
proposed such as normalisation techniques or limiting activation functions to
linear rectifying units. In this work we follow a different approach which is
particularly applicable to closed-loop learning of forward models where
back-propagation makes exclusive use of the sign of the error signal to prime
the learning, whilst a global relevance signal modulates the rate of learning.
This is inspired by the interaction between local plasticity and a global
neuromodulation. For example, whilst driving on an empty road, one can allow
for slow step-wise optimisation of actions, whereas, at a busy junction, an
error must be corrected at once. Hence, the error is the priming signal and the
intensity of the experience is a modulating factor in the weight change. The
advantages of this Prime and Modulate paradigm is twofold: it is free from
normalisation and it makes use of relevant cues from the environment to enrich
the learning. We present a mathematical derivation of the learning rule in
z-space and demonstrate the real-time performance with a robotic platform. The
results show a significant improvement in the speed of convergence compared to
that of the conventional back-propagation.