Enhancing Long-Term Recommendation with Bi-level Learnable Large
  Language Model Planning
Wentao Shi Xiangnan He Yang Zhang Chongming Gao Xinyue Li Jizhi Zhang Qifan Wang Fuli Feng
Traditional recommendation setting tends to excessively cater to users'
immediate interests and neglect their long-term engagement. To address it, it
is crucial to incorporate planning capabilities into the recommendation
decision-making process to develop policies that take into account both
immediate interests and long-term engagement. Despite Reinforcement Learning
(RL) can learn planning capacity by maximizing cumulative reward, the scarcity
of recommendation data presents challenges such as instability and
susceptibility to overfitting when training RL models from scratch.
  In this context, we propose to leverage the remarkable planning capabilities
over sparse data of Large Language Models (LLMs) for long-term recommendation.
The key lies in enabling a language model to understand and apply task-solving
principles effectively in personalized recommendation scenarios, as the model's
pre-training may not naturally encompass these principles, necessitating the
need to inspire or teach the model. To achieve this, we propose a Bi-level
Learnable LLM Planner framework, which combines macro-learning and
micro-learning through a hierarchical mechanism. The framework includes a
Planner and Reflector for acquiring high-level guiding principles and an
Actor-Critic component for planning personalization. Extensive experiments
validate the superiority of the framework in learning to plan for long-term
recommendations.