On the Non-Associativity of Analog Computations
Lisa Kuhn Bernhard Klein Holger Fröning
The energy efficiency of analog forms of computing makes it one of the most
promising candidates to deploy resource-hungry machine learning tasks on
resource-constrained system such as mobile or embedded devices. However, it is
well known that for analog computations the safety net of discretization is
missing, thus all analog computations are exposed to a variety of imperfections
of corresponding implementations. Examples include non-linearities, saturation
effect and various forms of noise. In this work, we observe that the ordering
of input operands of an analog operation also has an impact on the output
result, which essentially makes analog computations non-associative, even
though the underlying operation might be mathematically associative. We conduct
a simple test by creating a model of a real analog processor which captures
such ordering effects. With this model we assess the importance of ordering by
comparing the test accuracy of a neural network for keyword spotting, which is
trained based either on an ordered model, on a non-ordered variant, and on real
hardware. The results prove the existence of ordering effects as well as their
high impact, as neglecting ordering results in substantial accuracy drops.