Exosense: A Vision-Centric Scene Understanding System For Safe
  Exoskeleton Navigation
Jianeng Wang Matias Mattamala Christina Kassab Lintong Zhang Maurice Fallon
Exoskeletons for daily use by those with mobility impairments are being
developed. They will require accurate and robust scene understanding systems.
Current research has used vision to identify immediate terrain and geometric
obstacles, however these approaches are constrained to detections directly in
front of the user and are limited to classifying a finite range of terrain
types (e.g., stairs, ramps and level-ground). This paper presents Exosense, a
vision-centric scene understanding system which is capable of generating rich,
globally-consistent elevation maps, incorporating both semantic and terrain
traversability information. It features an elastic Atlas mapping framework
associated with a visual SLAM pose graph, embedded with open-vocabulary room
labels from a Vision-Language Model (VLM). The device's design includes a wide
field-of-view (FoV) fisheye multi-camera system to mitigate the challenges
introduced by the exoskeleton walking pattern. We demonstrate the system's
robustness to the challenges of typical periodic walking gaits, and its ability
to construct accurate semantically-rich maps in indoor settings. Additionally,
we showcase its potential for motion planning -- providing a step towards safe
navigation for exoskeletons.