Exploring CausalWorld: Enhancing robotic manipulation via knowledge
  transfer and curriculum learning
Xinrui Wang Yan Jin
This study explores a learning-based tri-finger robotic arm manipulating
task, which requires complex movements and coordination among the fingers. By
employing reinforcement learning, we train an agent to acquire the necessary
skills for proficient manipulation. To enhance the efficiency and effectiveness
of the learning process, two knowledge transfer strategies, fine-tuning and
curriculum learning, were utilized within the soft actor-critic architecture.
Fine-tuning allows the agent to leverage pre-trained knowledge and adapt it to
new tasks. Several variations like model transfer, policy transfer, and
across-task transfer were implemented and evaluated. To eliminate the need for
pretraining, curriculum learning decomposes the advanced task into simpler,
progressive stages, mirroring how humans learn. The number of learning stages,
the context of the sub-tasks, and the transition timing were found to be the
critical design parameters. The key factors of two learning strategies and
corresponding effects were explored in context-aware and context-unaware
scenarios, enabling us to identify the scenarios where the methods demonstrate
optimal performance, derive conclusive insights, and contribute to a broader
range of learning-based engineering applications.