COMPOSER: Scalable and Robust Modular Policies for Snake Robots
Yuyou Zhang Yaru Niu Xingyu Liu Ding Zhao
Snake robots have showcased remarkable compliance and adaptability in their
interaction with environments, mirroring the traits of their natural
counterparts. While their hyper-redundant and high-dimensional characteristics
add to this adaptability, they also pose great challenges to robot control.
Instead of perceiving the hyper-redundancy and flexibility of snake robots as
mere challenges, there lies an unexplored potential in leveraging these traits
to enhance robustness and generalizability at the control policy level. We seek
to develop a control policy that effectively breaks down the high
dimensionality of snake robots while harnessing their redundancy. In this work,
we consider the snake robot as a modular robot and formulate the control of the
snake robot as a cooperative Multi-Agent Reinforcement Learning (MARL) problem.
Each segment of the snake robot functions as an individual agent. Specifically,
we incorporate a self-attention mechanism to enhance the cooperative behavior
between agents. A high-level imagination policy is proposed to provide
additional rewards to guide the low-level control policy. We validate the
proposed method COMPOSER with five snake robot tasks, including goal reaching,
wall climbing, shape formation, tube crossing, and block pushing. COMPOSER
achieves the highest success rate across all tasks when compared to a
centralized baseline and four modular policy baselines. Additionally, we show
enhanced robustness against module corruption and significantly superior
zero-shot generalizability in our proposed method. The videos of this work are
available on our project page: https://sites.google.com/view/composer-snake/.