Cooperative Colorization: Exploring Latent Cross-Domain Priors for NIR
  Image Spectrum Translation
Xingxing Yang Jie Chen Zaifeng Yang
Near-infrared (NIR) image spectrum translation is a challenging problem with
many promising applications. Existing methods struggle with the mapping
ambiguity between the NIR and the RGB domains, and generalize poorly due to the
limitations of models' learning capabilities and the unavailability of
sufficient NIR-RGB image pairs for training. To address these challenges, we
propose a cooperative learning paradigm that colorizes NIR images in parallel
with another proxy grayscale colorization task by exploring latent cross-domain
priors (i.e., latent spectrum context priors and task domain priors), dubbed
CoColor. The complementary statistical and semantic spectrum information from
these two task domains -- in the forms of pre-trained colorization networks --
are brought in as task domain priors. A bilateral domain translation module is
subsequently designed, in which intermittent NIR images are generated from
grayscale and colorized in parallel with authentic NIR images; and vice versa
for the grayscale images. These intermittent transformations act as latent
spectrum context priors for efficient domain knowledge exchange. We
progressively fine-tune and fuse these modules with a series of pixel-level and
feature-level consistency constraints. Experiments show that our proposed
cooperative learning framework produces satisfactory spectrum translation
outputs with diverse colors and rich textures, and outperforms state-of-the-art
counterparts by 3.95dB and 4.66dB in terms of PNSR for the NIR and grayscale
colorization tasks, respectively.