Monadic Deep Learning
Bo Yang Zhihao Zhang Kirisame Marisa Kai Shi
The Java and Scala community has built a very successful big data ecosystem.
However, most of neural networks running on it are modeled in dynamically typed
programming languages. These dynamically typed deep learning frameworks treat
neural networks as differentiable expressions that contain many trainable
variable, and perform automatic differentiation on those expressions when
training them.
  Until 2019, none of the learning frameworks in statically typed languages
provided the expressive power of traditional frameworks. Their users are not
able to use custom algorithms unless creating plenty of boilerplate code for
hard-coded back-propagation.
  We solved this problem in DeepLearning.scala 2. Our contributions are:
  1. We discovered a novel approach to perform automatic differentiation in
reverse mode for statically typed functions that contain multiple trainable
variable, and can interoperate freely with the metalanguage.
  2. We designed a set of monads and monad transformers, which allow users to
create monadic expressions that represent dynamic neural networks.
  3. Along with these monads, we provide some applicative functors, to perform
multiple calculations in parallel.
  With these features, users of DeepLearning.scala were able to create complex
neural networks in an intuitive and concise way, and still maintain type
safety.