Addressing Discontinuous Root-Finding for Subsequent Differentiability
  in Machine Learning, Inverse Problems, and Control
Daniel Johnson Ronald Fedkiw
There are many physical processes that have inherent discontinuities in their
mathematical formulations. This paper is motivated by the specific case of
collisions between two rigid or deformable bodies and the intrinsic nature of
that discontinuity. The impulse response to a collision is discontinuous with
the lack of any response when no collision occurs, which causes difficulties
for numerical approaches that require differentiability which are typical in
machine learning, inverse problems, and control. We theoretically and
numerically demonstrate that the derivative of the collision time with respect
to the parameters becomes infinite as one approaches the barrier separating
colliding from not colliding, and use lifting to complexify the solution space
so that solutions on the other side of the barrier are directly attainable as
precise values. Subsequently, we mollify the barrier posed by the unbounded
derivatives, so that one can tunnel back and forth in a smooth and reliable
fashion facilitating the use of standard numerical approaches. Moreover, we
illustrate that standard approaches fail in numerous ways mostly due to a lack
of understanding of the mathematical nature of the problem (e.g. typical
backpropagation utilizes many rules of differentiation, but ignores L'Hopital's
rule).