Improving Lens Flare Removal with General Purpose Pipeline and Multiple
  Light Sources Recovery
Yuyan Zhou Dong Liang Songcan Chen Sheng-Jun Huang Shuo Yang Chongyi Li
When taking images against strong light sources, the resulting images often
contain heterogeneous flare artifacts. These artifacts can importantly affect
image visual quality and downstream computer vision tasks. While collecting
real data pairs of flare-corrupted/flare-free images for training flare removal
models is challenging, current methods utilize the direct-add approach to
synthesize data. However, these methods do not consider automatic exposure and
tone mapping in image signal processing pipeline (ISP), leading to the limited
generalization capability of deep models training using such data. Besides,
existing methods struggle to handle multiple light sources due to the different
sizes, shapes and illuminance of various light sources. In this paper, we
propose a solution to improve the performance of lens flare removal by
revisiting the ISP and remodeling the principle of automatic exposure in the
synthesis pipeline and design a more reliable light sources recovery strategy.
The new pipeline approaches realistic imaging by discriminating the local and
global illumination through convex combination, avoiding global illumination
shifting and local over-saturation. Our strategy for recovering multiple light
sources convexly averages the input and output of the neural network based on
illuminance levels, thereby avoiding the need for a hard threshold in
identifying light sources. We also contribute a new flare removal testing
dataset containing the flare-corrupted images captured by ten types of consumer
electronics. The dataset facilitates the verification of the generalization
capability of flare removal methods. Extensive experiments show that our
solution can effectively improve the performance of lens flare removal and push
the frontier toward more general situations.