Under-Display Camera Image Restoration with Scattering Effect
Binbin Song Xiangyu Chen Shuning Xu Jiantao Zhou
The under-display camera (UDC) provides consumers with a full-screen visual
experience without any obstruction due to notches or punched holes. However,
the semi-transparent nature of the display inevitably introduces the severe
degradation into UDC images. In this work, we address the UDC image restoration
problem with the specific consideration of the scattering effect caused by the
display. We explicitly model the scattering effect by treating the display as a
piece of homogeneous scattering medium. With the physical model of the
scattering effect, we improve the image formation pipeline for the image
synthesis to construct a realistic UDC dataset with ground truths. To suppress
the scattering effect for the eventual UDC image recovery, a two-branch
restoration network is designed. More specifically, the scattering branch
leverages global modeling capabilities of the channel-wise self-attention to
estimate parameters of the scattering effect from degraded images. While the
image branch exploits the local representation advantage of CNN to recover
clear scenes, implicitly guided by the scattering branch. Extensive experiments
are conducted on both real-world and synthesized data, demonstrating the
superiority of the proposed method over the state-of-the-art UDC restoration
techniques. The source code and dataset are available at
\url{https://github.com/NamecantbeNULL/SRUDC}.