Learning Restoration is Not Enough: Transfering Identical Mapping for
  Single-Image Shadow Removal
Xiaoguang Li Qing Guo Pingping Cai Wei Feng Ivor Tsang Song Wang
Shadow removal is to restore shadow regions to their shadow-free counterparts
while leaving non-shadow regions unchanged. State-of-the-art shadow removal
methods train deep neural networks on collected shadow & shadow-free image
pairs, which are desired to complete two distinct tasks via shared weights,
i.e., data restoration for shadow regions and identical mapping for non-shadow
regions. We find that these two tasks exhibit poor compatibility, and using
shared weights for these two tasks could lead to the model being optimized
towards only one task instead of both during the training process. Note that
such a key issue is not identified by existing deep learning-based shadow
removal methods. To address this problem, we propose to handle these two tasks
separately and leverage the identical mapping results to guide the shadow
restoration in an iterative manner. Specifically, our method consists of three
components: an identical mapping branch (IMB) for non-shadow regions
processing, an iterative de-shadow branch (IDB) for shadow regions restoration
based on identical results, and a smart aggregation block (SAB). The IMB aims
to reconstruct an image that is identical to the input one, which can benefit
the restoration of the non-shadow regions without explicitly distinguishing
between shadow and non-shadow regions. Utilizing the multi-scale features
extracted by the IMB, the IDB can effectively transfer information from
non-shadow regions to shadow regions progressively, facilitating the process of
shadow removal. The SAB is designed to adaptive integrate features from both
IMB and IDB. Moreover, it generates a finely tuned soft shadow mask that guides
the process of removing shadows. Extensive experiments demonstrate our method
outperforms all the state-of-the-art shadow removal approaches on the widely
used shadow removal datasets.