Event Camera Data Dense Pre-training
Yan Yang Liyuan Pan Liu Liu
This paper introduces a self-supervised learning framework designed for
pre-training neural networks tailored to dense prediction tasks using event
camera data. Our approach utilizes solely event data for training.
  Transferring achievements from dense RGB pre-training directly to event
camera data yields subpar performance. This is attributed to the spatial
sparsity inherent in an event image (converted from event data), where many
pixels do not contain information. To mitigate this sparsity issue, we encode
an event image into event patch features, automatically mine contextual
similarity relationships among patches, group the patch features into
distinctive contexts, and enforce context-to-context similarities to learn
discriminative event features.
  For training our framework, we curate a synthetic event camera dataset
featuring diverse scene and motion patterns. Transfer learning performance on
downstream dense prediction tasks illustrates the superiority of our method
over state-of-the-art approaches. Notably, our single model secured the top
position in the challenging DSEC-Flow benchmark.