X-maps: Direct Depth Lookup for Event-based Structured Light Systems
Wieland Morgenstern Niklas Gard Simon Baumann Anna Hilsmann Peter Eisert
We present a new approach to direct depth estimation for Spatial Augmented
Reality (SAR) applications using event cameras. These dynamic vision sensors
are a great fit to be paired with laser projectors for depth estimation in a
structured light approach. Our key contributions involve a conversion of the
projector time map into a rectified X-map, capturing x-axis correspondences for
incoming events and enabling direct disparity lookup without any additional
search. Compared to previous implementations, this significantly simplifies
depth estimation, making it more efficient, while the accuracy is similar to
the time map-based process. Moreover, we compensate non-linear temporal
behavior of cheap laser projectors by a simple time map calibration, resulting
in improved performance and increased depth estimation accuracy. Since depth
estimation is executed by two lookups only, it can be executed almost instantly
(less than 3 ms per frame with a Python implementation) for incoming events.
This allows for real-time interactivity and responsiveness, which makes our
approach especially suitable for SAR experiences where low latency, high frame
rates and direct feedback are crucial. We present valuable insights gained into
data transformed into X-maps and evaluate our depth from disparity estimation
against the state of the art time map-based results. Additional results and
code are available on our project page: https://fraunhoferhhi.github.io/X-maps/