Differentiable Visual Computing for Inverse Problems and Machine
  Learning
Andrew Spielberg Fangcheng Zhong Konstantinos Rematas Krishna Murthy Jatavallabhula Cengiz Oztireli Tzu-Mao Li Derek Nowrouzezahrai
Originally designed for applications in computer graphics, visual computing
(VC) methods synthesize information about physical and virtual worlds, using
prescribed algorithms optimized for spatial computing. VC is used to analyze
geometry, physically simulate solids, fluids, and other media, and render the
world via optical techniques. These fine-tuned computations that operate
explicitly on a given input solve so-called forward problems, VC excels at. By
contrast, deep learning (DL) allows for the construction of general algorithmic
models, side stepping the need for a purely first principles-based approach to
problem solving. DL is powered by highly parameterized neural network
architectures -- universal function approximators -- and gradient-based search
algorithms which can efficiently search that large parameter space for optimal
models. This approach is predicated by neural network differentiability, the
requirement that analytic derivatives of a given problem's task metric can be
computed with respect to neural network's parameters. Neural networks excel
when an explicit model is not known, and neural network training solves an
inverse problem in which a model is computed from data.