Software-based Automatic Differentiation is Flawed
Daniel Johnson Trevor Maxfield Yongxu Jin Ronald Fedkiw
Various software efforts embrace the idea that object oriented programming
enables a convenient implementation of the chain rule, facilitating so-called
automatic differentiation via backpropagation. Such frameworks have no
mechanism for simplifying the expressions (obtained via the chain rule) before
evaluating them. As we illustrate below, the resulting errors tend to be
unbounded.