Multiple Key-value Strategy in Recommendation Systems Incorporating
  Large Language Model
Dui Wang Xiangyu Hou Xiaohui Yang Bo Zhang Renbing Chen Daiyue Xue
Recommendation system (RS) plays significant roles in matching users
information needs for Internet applications, and it usually utilizes the
vanilla neural network as the backbone to handle embedding details. Recently,
the large language model (LLM) has exhibited emergent abilities and achieved
great breakthroughs both in the CV and NLP communities. Thus, it is logical to
incorporate RS with LLM better, which has become an emerging research
direction. Although some existing works have made their contributions to this
issue, they mainly consider the single key situation (e.g. historical
interactions), especially in sequential recommendation. The situation of
multiple key-value data is simply neglected. This significant scenario is
mainstream in real practical applications, where the information of users (e.g.
age, occupation, etc) and items (e.g. title, category, etc) has more than one
key. Therefore, we aim to implement sequential recommendations based on
multiple key-value data by incorporating RS with LLM. In particular, we
instruct tuning a prevalent open-source LLM (Llama 7B) in order to inject
domain knowledge of RS into the pre-trained LLM. Since we adopt multiple
key-value strategies, LLM is hard to learn well among these keys. Thus the
general and innovative shuffle and mask strategies, as an innovative manner of
data argument, are designed. To demonstrate the effectiveness of our approach,
extensive experiments are conducted on the popular and suitable dataset
MovieLens which contains multiple keys-value. The experimental results
demonstrate that our approach can nicely and effectively complete this
challenging issue.