Relating Events and Frames Based on Self-Supervised Learning and
  Uncorrelated Conditioning for Unsupervised Domain Adaptation
Mohammad Rostami Dayuan Jian
Event-based cameras provide accurate and high temporal resolution
measurements for performing computer vision tasks in challenging scenarios,
such as high-dynamic range environments and fast-motion maneuvers. Despite
their advantages, utilizing deep learning for event-based vision encounters a
significant obstacle due to the scarcity of annotated data caused by the
relatively recent emergence of event-based cameras. To overcome this
limitation, leveraging the knowledge available from annotated data obtained
with conventional frame-based cameras presents an effective solution based on
unsupervised domain adaptation. We propose a new algorithm tailored for
adapting a deep neural network trained on annotated frame-based data to
generalize well on event-based unannotated data. Our approach incorporates
uncorrelated conditioning and self-supervised learning in an adversarial
learning scheme to close the gap between the two source and target domains. By
applying self-supervised learning, the algorithm learns to align the
representations of event-based data with those from frame-based camera data,
thereby facilitating knowledge transfer.Furthermore, the inclusion of
uncorrelated conditioning ensures that the adapted model effectively
distinguishes between event-based and conventional data, enhancing its ability
to classify event-based images accurately.Through empirical experimentation and
evaluation, we demonstrate that our algorithm surpasses existing approaches
designed for the same purpose using two benchmarks. The superior performance of
our solution is attributed to its ability to effectively utilize annotated data
from frame-based cameras and transfer the acquired knowledge to the event-based
vision domain.