SeNM-VAE: Semi-Supervised Noise Modeling with Hierarchical Variational
  Autoencoder
Dihan Zheng Yihang Zou Xiaowen Zhang Chenglong Bao
The data bottleneck has emerged as a fundamental challenge in learning based
image restoration methods. Researchers have attempted to generate synthesized
training data using paired or unpaired samples to address this challenge. This
study proposes SeNM-VAE, a semi-supervised noise modeling method that leverages
both paired and unpaired datasets to generate realistic degraded data. Our
approach is based on modeling the conditional distribution of degraded and
clean images with a specially designed graphical model. Under the variational
inference framework, we develop an objective function for handling both paired
and unpaired data. We employ our method to generate paired training samples for
real-world image denoising and super-resolution tasks. Our approach excels in
the quality of synthetic degraded images compared to other unpaired and paired
noise modeling methods. Furthermore, our approach demonstrates remarkable
performance in downstream image restoration tasks, even with limited paired
data. With more paired data, our method achieves the best performance on the
SIDD dataset.