Masked Pre-trained Model Enables Universal Zero-shot Denoiser
Xiaoxiao Ma Zhixiang Wei Yi Jin Pengyang Ling Tianle Liu Ben Wang Junkang Dai Huaian Chen Enhong Chen
In this work, we observe that the model, which is trained on vast general
images using masking strategy, has been naturally embedded with the
distribution knowledge regarding natural images, and thus spontaneously attains
the underlying potential for strong image denoising. Based on this observation,
we propose a novel zero-shot denoising paradigm, i.e., Masked Pre-train then
Iterative fill (MPI). MPI pre-trains a model with masking and fine-tunes it for
denoising of a single image with unseen noise degradation. Concretely, the
proposed MPI comprises two key procedures: 1) Masked Pre-training involves
training a model on multiple natural images with random masks to gather
generalizable representations, allowing for practical applications in varying
noise degradation and even in distinct image types. 2) Iterative filling is
devised to efficiently fuse pre-trained knowledge for denoising. Similar to but
distinct from pre-training, random masking is retained to bridge the gap, but
only the predicted parts covered by masks are assembled for efficiency, which
enables high-quality denoising within a limited number of iterations.
Comprehensive experiments across various noisy scenarios underscore the notable
advances of proposed MPI over previous approaches with a marked reduction in
inference time. Code is available at https://github.com/krennic999/MPI.git.