Physics-Driven Turbulence Image Restoration with Stochastic Refinement
Ajay Jaiswal Xingguang Zhang Stanley H. Chan Zhangyang Wang
Image distortion by atmospheric turbulence is a stochastic degradation, which
is a critical problem in long-range optical imaging systems. A number of
research has been conducted during the past decades, including model-based and
emerging deep-learning solutions with the help of synthetic data. Although fast
and physics-grounded simulation tools have been introduced to help the
deep-learning models adapt to real-world turbulence conditions recently, the
training of such models only relies on the synthetic data and ground truth
pairs. This paper proposes the Physics-integrated Restoration Network (PiRN) to
bring the physics-based simulator directly into the training process to help
the network to disentangle the stochasticity from the degradation and the
underlying image. Furthermore, to overcome the ``average effect" introduced by
deterministic models and the domain gap between the synthetic and real-world
degradation, we further introduce PiRN with Stochastic Refinement (PiRN-SR) to
boost its perceptual quality. Overall, our PiRN and PiRN-SR improve the
generalization to real-world unknown turbulence conditions and provide a
state-of-the-art restoration in both pixel-wise accuracy and perceptual
quality. Our codes are available at \url{https://github.com/VITA-Group/PiRN}.