SPA: Towards A Computational Friendly Cloud-Base and On-Devices
  Collaboration Seq2seq Personalized Generation
Yanming Liu Xinyue Peng Jiannan Cao Le Dai Xingzu Liu Weihao Liu Mingbang Wang
Large language models(LLMs) have shown its outperforming ability on various
tasks and question answering. However, LLMs require high computation cost and
large memory cost. At the same time, LLMs may cause privacy leakage when
training or prediction procedure contains sensitive information. In this paper,
we propose SPA(Side Plugin Adaption), a lightweight architecture for fast
on-devices inference and privacy retaining on the constraints of strict
on-devices computation and memory constraints. Compared with other on-devices
seq2seq generation, SPA could make a fast and stable inference on low-resource
constraints, allowing it to obtain cost effiency. Our method establish an
interaction between a pretrained LLMs on-cloud and additive parameters
on-devices, which could provide the knowledge on both pretrained LLMs and
private personal feature.Further more, SPA provides a framework to keep
feature-base parameters on private guaranteed but low computational devices
while leave the parameters containing general information on the high
computational devices.