Addressing Imperfect Symmetry: a Novel Symmetry-Learning Actor-Critic
  Extension
Miguel Abreu Luis Paulo Reis Nuno Lau
Symmetry, a fundamental concept to understand our environment, often
oversimplifies reality from a mathematical perspective. Humans are a prime
example, deviating from perfect symmetry in terms of appearance and cognitive
biases (e.g. having a dominant hand). Nevertheless, our brain can easily
overcome these imperfections and efficiently adapt to symmetrical tasks. The
driving motivation behind this work lies in capturing this ability through
reinforcement learning. To this end, we introduce Adaptive Symmetry Learning
(ASL) $\unicode{x2013}$ a model-minimization actor-critic extension that
addresses incomplete or inexact symmetry descriptions by adapting itself during
the learning process. ASL consists of a symmetry fitting component and a
modular loss function that enforces a common symmetric relation across all
states while adapting to the learned policy. The performance of ASL is compared
to existing symmetry-enhanced methods in a case study involving a four-legged
ant model for multidirectional locomotion tasks. The results demonstrate that
ASL is capable of recovering from large perturbations and generalizing
knowledge to hidden symmetric states. It achieves comparable or better
performance than alternative methods in most scenarios, making it a valuable
approach for leveraging model symmetry while compensating for inherent
perturbations.