Temporal-Spatial Processing of Event Camera Data via Delay-Loop
  Reservoir Neural Network
Richard Lau Anthony Tylan-Tyler Lihan Yao Rey de Castro Roberto Robert Taylor Isaiah Jones
This paper describes a temporal-spatial model for video processing with
special applications to processing event camera videos. We propose to study a
conjecture motivated by our previous study of video processing with delay loop
reservoir (DLR) neural network, which we call Temporal-Spatial Conjecture
(TSC). The TSC postulates that there is significant information content carried
in the temporal representation of a video signal and that machine learning
algorithms would benefit from separate optimization of the spatial and temporal
components for intelligent processing. To verify or refute the TSC, we propose
a Visual Markov Model (VMM) which decompose the video into spatial and temporal
components and estimate the mutual information (MI) of these components. Since
computation of video mutual information is complex and time consuming, we use a
Mutual Information Neural Network to estimate the bounds of the mutual
information. Our result shows that the temporal component carries significant
MI compared to that of the spatial component. This finding has often been
overlooked in neural network literature. In this paper, we will exploit this
new finding to guide our design of a delay-loop reservoir neural network for
event camera classification, which results in a 18% improvement on
classification accuracy.