High-Degrees-of-Freedom Dynamic Neural Fields for Robot Self-Modeling
  and Motion Planning
Lennart Schulze Hod Lipson
A robot self-model is a task-agnostic representation of the robot's physical
morphology that can be used for motion planning tasks in absence of classical
geometric kinematic models. In particular, when the latter are hard to engineer
or the robot's kinematics change unexpectedly, human-free self-modeling is a
necessary feature of truly autonomous agents. In this work, we leverage neural
fields to allow a robot to self-model its kinematics as a neural-implicit query
model learned only from 2D images annotated with camera poses and
configurations. This enables significantly greater applicability than existing
approaches which have been dependent on depth images or geometry knowledge. To
this end, alongside a curricular data sampling strategy, we propose a new
encoder-based neural density field architecture for dynamic object-centric
scenes conditioned on high numbers of degrees of freedom (DOFs). In a 7-DOF
robot test setup, the learned self-model achieves a Chamfer-L2 distance of 2%
of the robot's workspace dimension. We demonstrate the capabilities of this
model on a motion planning task as an exemplary downstream application.