Tiny Reinforcement Learning for Quadruped Locomotion using Decision
  Transformers
Orhan Eren Akgün Néstor Cuevas Matheus Farias Daniel Garces
Resource-constrained robotic platforms are particularly useful for tasks that
require low-cost hardware alternatives due to the risk of losing the robot,
like in search-and-rescue applications, or the need for a large number of
devices, like in swarm robotics. For this reason, it is crucial to find
mechanisms for adapting reinforcement learning techniques to the constraints
imposed by lower computational power and smaller memory capacities of these
ultra low-cost robotic platforms. We try to address this need by proposing a
method for making imitation learning deployable onto resource-constrained
robotic platforms. Here we cast the imitation learning problem as a conditional
sequence modeling task and we train a decision transformer using expert
demonstrations augmented with a custom reward. Then, we compress the resulting
generative model using software optimization schemes, including quantization
and pruning. We test our method in simulation using Isaac Gym, a realistic
physics simulation environment designed for reinforcement learning. We
empirically demonstrate that our method achieves natural looking gaits for
Bittle, a resource-constrained quadruped robot. We also run multiple
simulations to show the effects of pruning and quantization on the performance
of the model. Our results show that quantization (down to 4 bits) and pruning
reduce model size by around 30\% while maintaining a competitive reward, making
the model deployable in a resource-constrained system.