Efficient Multi-View Inverse Rendering Using a Hybrid Differentiable
  Rendering Method
Xiangyang Zhu Yiling Pan Bailin Deng Bin Wang
Recovering the shape and appearance of real-world objects from natural 2D
images is a long-standing and challenging inverse rendering problem. In this
paper, we introduce a novel hybrid differentiable rendering method to
efficiently reconstruct the 3D geometry and reflectance of a scene from
multi-view images captured by conventional hand-held cameras. Our method
follows an analysis-by-synthesis approach and consists of two phases. In the
initialization phase, we use traditional SfM and MVS methods to reconstruct a
virtual scene roughly matching the real scene. Then in the optimization phase,
we adopt a hybrid approach to refine the geometry and reflectance, where the
geometry is first optimized using an approximate differentiable rendering
method, and the reflectance is optimized afterward using a physically-based
differentiable rendering method. Our hybrid approach combines the efficiency of
approximate methods with the high-quality results of physically-based methods.
Extensive experiments on synthetic and real data demonstrate that our method
can produce reconstructions with similar or higher quality than
state-of-the-art methods while being more efficient.