Retentive Decision Transformer with Adaptive Masking for Reinforcement
  Learning based Recommendation Systems
Siyu Wang Xiaocong Chen Lina Yao
Reinforcement Learning-based Recommender Systems (RLRS) have shown promise
across a spectrum of applications, from e-commerce platforms to streaming
services. Yet, they grapple with challenges, notably in crafting reward
functions and harnessing large pre-existing datasets within the RL framework.
Recent advancements in offline RLRS provide a solution for how to address these
two challenges. However, existing methods mainly rely on the transformer
architecture, which, as sequence lengths increase, can introduce challenges
associated with computational resources and training costs. Additionally, the
prevalent methods employ fixed-length input trajectories, restricting their
capacity to capture evolving user preferences. In this study, we introduce a
new offline RLRS method to deal with the above problems. We reinterpret the
RLRS challenge by modeling sequential decision-making as an inference task,
leveraging adaptive masking configurations. This adaptive approach selectively
masks input tokens, transforming the recommendation task into an inference
challenge based on varying token subsets, thereby enhancing the agent's ability
to infer across diverse trajectory lengths. Furthermore, we incorporate a
multi-scale segmented retention mechanism that facilitates efficient modeling
of long sequences, significantly enhancing computational efficiency. Our
experimental analysis, conducted on both online simulator and offline datasets,
clearly demonstrates the advantages of our proposed method.