MIMONets: Multiple-Input-Multiple-Output Neural Networks Exploiting
  Computation in Superposition
Nicolas Menet Michael Hersche Geethan Karunaratne Luca Benini Abu Sebastian Abbas Rahimi
With the advent of deep learning, progressively larger neural networks have
been designed to solve complex tasks. We take advantage of these capacity-rich
models to lower the cost of inference by exploiting computation in
superposition. To reduce the computational burden per input, we propose
Multiple-Input-Multiple-Output Neural Networks (MIMONets) capable of handling
many inputs at once. MIMONets augment various deep neural network architectures
with variable binding mechanisms to represent an arbitrary number of inputs in
a compositional data structure via fixed-width distributed representations.
Accordingly, MIMONets adapt nonlinear neural transformations to process the
data structure holistically, leading to a speedup nearly proportional to the
number of superposed input items in the data structure. After processing in
superposition, an unbinding mechanism recovers each transformed input of
interest. MIMONets also provide a dynamic trade-off between accuracy and
throughput by an instantaneous on-demand switching between a set of
accuracy-throughput operating points, yet within a single set of fixed
parameters. We apply the concept of MIMONets to both CNN and Transformer
architectures resulting in MIMOConv and MIMOFormer, respectively. Empirical
evaluations show that MIMOConv achieves about 2-4 x speedup at an accuracy
delta within [+0.68, -3.18]% compared to WideResNet CNNs on CIFAR10 and
CIFAR100. Similarly, MIMOFormer can handle 2-4 inputs at once while maintaining
a high average accuracy within a [-1.07, -3.43]% delta on the long range arena
benchmark. Finally, we provide mathematical bounds on the interference between
superposition channels in MIMOFormer. Our code is available at
https://github.com/IBM/multiple-input-multiple-output-nets.