Rawformer: Unpaired Raw-to-Raw Translation for Learnable Camera ISPs
Georgy Perevozchikov Nancy Mehta Mahmoud Afifi Radu Timofte
Modern smartphone camera quality heavily relies on the image signal processor
(ISP) to enhance captured raw images, utilizing carefully designed modules to
produce final output images encoded in a standard color space (e.g., sRGB).
Neural-based end-to-end learnable ISPs offer promising advancements,
potentially replacing traditional ISPs with their ability to adapt without
requiring extensive tuning for each new camera model, as is often the case for
nearly every module in traditional ISPs. However, the key challenge with the
recent learning-based ISPs is the urge to collect large paired datasets for
each distinct camera model due to the influence of intrinsic camera
characteristics on the formation of input raw images. This paper tackles this
challenge by introducing a novel method for unpaired learning of raw-to-raw
translation across diverse cameras. Specifically, we propose Rawformer, an
unsupervised Transformer-based encoder-decoder method for raw-to-raw
translation. It accurately maps raw images captured by a certain camera to the
target camera, facilitating the generalization of learnable ISPs to new unseen
cameras. Our method demonstrates superior performance on real camera datasets,
achieving higher accuracy compared to previous state-of-the-art techniques, and
preserving a more robust correlation between the original and translated raw
images.