Unsupervised Intrinsic Image Decomposition with LiDAR Intensity Enhanced
  Training
Shogo Sato Takuhiro Kaneko Kazuhiko Murasaki Taiga Yoshida Ryuichi Tanida Akisato Kimura
Unsupervised intrinsic image decomposition (IID) is the process of separating
a natural image into albedo and shade without these ground truths. A recent
model employing light detection and ranging (LiDAR) intensity demonstrated
impressive performance, though the necessity of LiDAR intensity during
inference restricts its practicality. Thus, IID models employing only a single
image during inference while keeping as high IID quality as the one with an
image plus LiDAR intensity are highly desired. To address this challenge, we
propose a novel approach that utilizes only an image during inference while
utilizing an image and LiDAR intensity during training. Specifically, we
introduce a partially-shared model that accepts an image and LiDAR intensity
individually using a different specific encoder but processes them together in
specific components to learn shared representations. In addition, to enhance
IID quality, we propose albedo-alignment loss and image-LiDAR conversion (ILC)
paths. Albedo-alignment loss aligns the gray-scale albedo from an image to that
inferred from LiDAR intensity, thereby reducing cast shadows in albedo from an
image due to the absence of cast shadows in LiDAR intensity. Furthermore, to
translate the input image into albedo and shade style while keeping the image
contents, the input image is separated into style code and content code by
encoders. The ILC path mutually translates the image and LiDAR intensity, which
share content but differ in style, contributing to the distinct differentiation
of style from content. Consequently, LIET achieves comparable IID quality to
the existing model with LiDAR intensity, while utilizing only an image without
LiDAR intensity during inference.