Language-Conditioned Path Planning
Amber Xie Youngwoon Lee Pieter Abbeel Stephen James
Contact is at the core of robotic manipulation. At times, it is desired (e.g.
manipulation and grasping), and at times, it is harmful (e.g. when avoiding
obstacles). However, traditional path planning algorithms focus solely on
collision-free paths, limiting their applicability in contact-rich tasks. To
address this limitation, we propose the domain of Language-Conditioned Path
Planning, where contact-awareness is incorporated into the path planning
problem. As a first step in this domain, we propose Language-Conditioned
Collision Functions (LACO) a novel approach that learns a collision function
using only a single-view image, language prompt, and robot configuration. LACO
predicts collisions between the robot and the environment, enabling flexible,
conditional path planning without the need for manual object annotations, point
cloud data, or ground-truth object meshes. In both simulation and the real
world, we demonstrate that LACO can facilitate complex, nuanced path plans that
allow for interaction with objects that are safe to collide, rather than
prohibiting any collision.