Exploring the Common Appearance-Boundary Adaptation for Nighttime
  Optical Flow
Hanyu Zhou Yi Chang Haoyue Liu Wending Yan Yuxing Duan Zhiwei Shi Luxin Yan
We investigate a challenging task of nighttime optical flow, which suffers
from weakened texture and amplified noise. These degradations weaken
discriminative visual features, thus causing invalid motion feature matching.
Typically, existing methods employ domain adaptation to transfer knowledge from
auxiliary domain to nighttime domain in either input visual space or output
motion space. However, this direct adaptation is ineffective, since there
exists a large domain gap due to the intrinsic heterogeneous nature of the
feature representations between auxiliary and nighttime domains. To overcome
this issue, we explore a common-latent space as the intermediate bridge to
reinforce the feature alignment between auxiliary and nighttime domains. In
this work, we exploit two auxiliary daytime and event domains, and propose a
novel common appearance-boundary adaptation framework for nighttime optical
flow. In appearance adaptation, we employ the intrinsic image decomposition to
embed the auxiliary daytime image and the nighttime image into a
reflectance-aligned common space. We discover that motion distributions of the
two reflectance maps are very similar, benefiting us to consistently transfer
motion appearance knowledge from daytime to nighttime domain. In boundary
adaptation, we theoretically derive the motion correlation formula between
nighttime image and accumulated events within a spatiotemporal gradient-aligned
common space. We figure out that the correlation of the two spatiotemporal
gradient maps shares significant discrepancy, benefitting us to contrastively
transfer boundary knowledge from event to nighttime domain. Moreover,
appearance adaptation and boundary adaptation are complementary to each other,
since they could jointly transfer global motion and local boundary knowledge to
the nighttime domain.