StyleGAN knows Normal, Depth, Albedo, and More
Anand Bhattad Daniel McKee Derek Hoiem D. A. Forsyth
Intrinsic images, in the original sense, are image-like maps of scene
properties like depth, normal, albedo or shading. This paper demonstrates that
StyleGAN can easily be induced to produce intrinsic images. The procedure is
straightforward. We show that, if StyleGAN produces $G({w})$ from latents
${w}$, then for each type of intrinsic image, there is a fixed offset ${d}_c$
so that $G({w}+{d}_c)$ is that type of intrinsic image for $G({w})$. Here
${d}_c$ is {\em independent of ${w}$}. The StyleGAN we used was pretrained by
others, so this property is not some accident of our training regime. We show
that there are image transformations StyleGAN will {\em not} produce in this
fashion, so StyleGAN is not a generic image regression engine.
  It is conceptually exciting that an image generator should ``know'' and
represent intrinsic images. There may also be practical advantages to using a
generative model to produce intrinsic images. The intrinsic images obtained
from StyleGAN compare well both qualitatively and quantitatively with those
obtained by using SOTA image regression techniques; but StyleGAN's intrinsic
images are robust to relighting effects, unlike SOTA methods.