Non-negative isomorphic neural networks for photonic neuromorphic
  accelerators
Manos Kirtas Nikolaos Passalis Nikolaos Pleros Anastasios Tefas
Neuromorphic photonic accelerators are becoming increasingly popular, since
they can significantly improve computation speed and energy efficiency, leading
to femtojoule per MAC efficiency. However, deploying existing DL models on such
platforms is not trivial, since a great range of photonic neural network
architectures relies on incoherent setups and power addition operational
schemes that cannot natively represent negative quantities. This results in
additional hardware complexity that increases cost and reduces energy
efficiency. To overcome this, we can train non-negative neural networks and
potentially exploit the full range of incoherent neuromorphic photonic
capabilities. However, existing approaches cannot achieve the same level of
accuracy as their regular counterparts, due to training difficulties, as also
recent evidence suggests. To this end, we introduce a methodology to obtain the
non-negative isomorphic equivalents of regular neural networks that meet
requirements of neuromorphic hardware, overcoming the aforementioned
limitations. Furthermore, we also introduce a sign-preserving optimization
approach that enables training of such isomorphic networks in a non-negative
manner.