Color Equivariant Convolutional Networks
Attila Lengyel Ombretta Strafforello Robert-Jan Bruintjes Alexander Gielisse Jan van Gemert
Color is a crucial visual cue readily exploited by Convolutional Neural
Networks (CNNs) for object recognition. However, CNNs struggle if there is data
imbalance between color variations introduced by accidental recording
conditions. Color invariance addresses this issue but does so at the cost of
removing all color information, which sacrifices discriminative power. In this
paper, we propose Color Equivariant Convolutions (CEConvs), a novel deep
learning building block that enables shape feature sharing across the color
spectrum while retaining important color information. We extend the notion of
equivariance from geometric to photometric transformations by incorporating
parameter sharing over hue-shifts in a neural network. We demonstrate the
benefits of CEConvs in terms of downstream performance to various tasks and
improved robustness to color changes, including train-test distribution shifts.
Our approach can be seamlessly integrated into existing architectures, such as
ResNets, and offers a promising solution for addressing color-based domain
shifts in CNNs.