Tree-Structured Shading Decomposition
Chen Geng Hong-Xing Yu Sharon Zhang Maneesh Agrawala Jiajun Wu
We study inferring a tree-structured representation from a single image for
object shading. Prior work typically uses the parametric or measured
representation to model shading, which is neither interpretable nor easily
editable. We propose using the shade tree representation, which combines basic
shading nodes and compositing methods to factorize object surface shading. The
shade tree representation enables novice users who are unfamiliar with the
physical shading process to edit object shading in an efficient and intuitive
manner. A main challenge in inferring the shade tree is that the inference
problem involves both the discrete tree structure and the continuous parameters
of the tree nodes. We propose a hybrid approach to address this issue. We
introduce an auto-regressive inference model to generate a rough estimation of
the tree structure and node parameters, and then we fine-tune the inferred
shade tree through an optimization algorithm. We show experiments on synthetic
images, captured reflectance, real images, and non-realistic vector drawings,
allowing downstream applications such as material editing, vectorized shading,
and relighting. Project website: https://chen-geng.com/inv-shade-trees