JSTR: Joint Spatio-Temporal Reasoning for Event-based Moving Object
  Detection
Hanyu Zhou Zhiwei Shi Hao Dong Shihan Peng Yi Chang Luxin Yan
Event-based moving object detection is a challenging task, where static
background and moving object are mixed together. Typically, existing methods
mainly align the background events to the same spatial coordinate system via
motion compensation to distinguish the moving object. However, they neglect the
potential spatial tailing effect of moving object events caused by excessive
motion, which may affect the structure integrity of the extracted moving
object. We discover that the moving object has a complete columnar structure in
the point cloud composed of motion-compensated events along the timestamp.
Motivated by this, we propose a novel joint spatio-temporal reasoning method
for event-based moving object detection. Specifically, we first compensate the
motion of background events using inertial measurement unit. In spatial
reasoning stage, we project the compensated events into the same image
coordinate, discretize the timestamp of events to obtain a time image that can
reflect the motion confidence, and further segment the moving object through
adaptive threshold on the time image. In temporal reasoning stage, we construct
the events into a point cloud along timestamp, and use RANSAC algorithm to
extract the columnar shape in the cloud for peeling off the background.
Finally, we fuse the results from the two reasoning stages to extract the final
moving object region. This joint spatio-temporal reasoning framework can
effectively detect the moving object from motion confidence and geometric
structure. Moreover, we conduct extensive experiments on various datasets to
verify that the proposed method can improve the moving object detection
accuracy by 13\%.