Offline Multitask Representation Learning for Reinforcement Learning
Haque Ishfaq Thanh Nguyen-Tang Songtao Feng Raman Arora Mengdi Wang Ming Yin Doina Precup
We study offline multitask representation learning in reinforcement learning
(RL), where a learner is provided with an offline dataset from different tasks
that share a common representation and is asked to learn the shared
representation. We theoretically investigate offline multitask low-rank RL, and
propose a new algorithm called MORL for offline multitask representation
learning. Furthermore, we examine downstream RL in reward-free, offline and
online scenarios, where a new task is introduced to the agent that shares the
same representation as the upstream offline tasks. Our theoretical results
demonstrate the benefits of using the learned representation from the upstream
offline task instead of directly learning the representation of the low-rank
model.