Continual All-in-One Adverse Weather Removal with Knowledge Replay on a
  Unified Network Structure
De Cheng Yanling Ji Dong Gong Yan Li Nannan Wang Junwei Han Dingwen Zhang
In real-world applications, image degeneration caused by adverse weather is
always complex and changes with different weather conditions from days and
seasons. Systems in real-world environments constantly encounter adverse
weather conditions that are not previously observed. Therefore, it practically
requires adverse weather removal models to continually learn from incrementally
collected data reflecting various degeneration types. Existing adverse weather
removal approaches, for either single or multiple adverse weathers, are mainly
designed for a static learning paradigm, which assumes that the data of all
types of degenerations to handle can be finely collected at one time before a
single-phase learning process. They thus cannot directly handle the incremental
learning requirements. To address this issue, we made the earliest effort to
investigate the continual all-in-one adverse weather removal task, in a setting
closer to real-world applications. Specifically, we develop a novel continual
learning framework with effective knowledge replay (KR) on a unified network
structure. Equipped with a principal component projection and an effective
knowledge distillation mechanism, the proposed KR techniques are tailored for
the all-in-one weather removal task. It considers the characteristics of the
image restoration task with multiple degenerations in continual learning, and
the knowledge for different degenerations can be shared and accumulated in the
unified network structure. Extensive experimental results demonstrate the
effectiveness of the proposed method to deal with this challenging task, which
performs competitively to existing dedicated or joint training image
restoration methods. Our code is available at
https://github.com/xiaojihh/CL_all-in-one.