PromptRR: Diffusion Models as Prompt Generators for Single Image
  Reflection Removal
Tao Wang Wanglong Lu Kaihao Zhang Wenhan Luo Tae-Kyun Kim Tong Lu Hongdong Li Ming-Hsuan Yang
Existing single image reflection removal (SIRR) methods using deep learning
tend to miss key low-frequency (LF) and high-frequency (HF) differences in
images, affecting their effectiveness in removing reflections. To address this
problem, this paper proposes a novel prompt-guided reflection removal
(PromptRR) framework that uses frequency information as new visual prompts for
better reflection performance. Specifically, the proposed framework decouples
the reflection removal process into the prompt generation and subsequent
prompt-guided restoration. For the prompt generation, we first propose a prompt
pre-training strategy to train a frequency prompt encoder that encodes the
ground-truth image into LF and HF prompts. Then, we adopt diffusion models
(DMs) as prompt generators to generate the LF and HF prompts estimated by the
pre-trained frequency prompt encoder. For the prompt-guided restoration, we
integrate specially generated prompts into the PromptFormer network, employing
a novel Transformer-based prompt block to effectively steer the model toward
enhanced reflection removal. The results on commonly used benchmarks show that
our method outperforms state-of-the-art approaches. The codes and models are
available at https://github.com/TaoWangzj/PromptRR.