V2CE: Video to Continuous Events Simulator
Zhongyang Zhang Shuyang Cui Kaidong Chai Haowen Yu Subhasis Dasgupta Upal Mahbub Tauhidur Rahman
Dynamic Vision Sensor (DVS)-based solutions have recently garnered
significant interest across various computer vision tasks, offering notable
benefits in terms of dynamic range, temporal resolution, and inference speed.
However, as a relatively nascent vision sensor compared to Active Pixel Sensor
(APS) devices such as RGB cameras, DVS suffers from a dearth of ample labeled
datasets. Prior efforts to convert APS data into events often grapple with
issues such as a considerable domain shift from real events, the absence of
quantified validation, and layering problems within the time axis. In this
paper, we present a novel method for video-to-events stream conversion from
multiple perspectives, considering the specific characteristics of DVS. A
series of carefully designed losses helps enhance the quality of generated
event voxels significantly. We also propose a novel local dynamic-aware
timestamp inference strategy to accurately recover event timestamps from event
voxels in a continuous fashion and eliminate the temporal layering problem.
Results from rigorous validation through quantified metrics at all stages of
the pipeline establish our method unquestionably as the current
state-of-the-art (SOTA).