SANPO: A Scene Understanding, Accessibility, Navigation, Pathfinding,
  Obstacle Avoidance Dataset
Sagar M. Waghmare Kimberly Wilber Dave Hawkey Xuan Yang Matthew Wilson Stephanie Debats Cattalyya Nuengsigkapian Astuti Sharma Lars Pandikow Huisheng Wang Hartwig Adam Mikhail Sirotenko
We introduce SANPO, a large-scale egocentric video dataset focused on dense
prediction in outdoor environments. It contains stereo video sessions collected
across diverse outdoor environments, as well as rendered synthetic video
sessions. (Synthetic data was provided by Parallel Domain.) All sessions have
(dense) depth and odometry labels. All synthetic sessions and a subset of real
sessions have temporally consistent dense panoptic segmentation labels. To our
knowledge, this is the first human egocentric video dataset with both large
scale dense panoptic segmentation and depth annotations. In addition to the
dataset we also provide zero-shot baselines and SANPO benchmarks for future
research. We hope that the challenging nature of SANPO will help advance the
state-of-the-art in video segmentation, depth estimation, multi-task visual
modeling, and synthetic-to-real domain adaptation, while enabling human
navigation systems.
  SANPO is available here:
https://google-research-datasets.github.io/sanpo_dataset/