GPTA: Generative Prompt Tuning Assistant for Synergistic Downstream
  Neural Network Enhancement with LLMs
Xiao Liu Jiawei Zhang
This study introduces GPTA, a Large Language Model assistance training
framework, that enhances the training of downstream task models via prefix
prompt. By minimizing data exposure to LLM, the framework addresses the
security and legal challenges of applying LLM in downstream task model
training. GPTA utilizes a new synergistic training approach, optimizing the
downstream models with parameter gradients and LLMs with the novel ``dialogue
gradient''. The framework not only demonstrates significant improvements in
model performance across six NLP benchmark datasets, but also reduces
overfitting in low-resource scenarios effectively. The detailed analyses
further validate that our pioneer framework provides a cost-efficient and
adaptive method for downstream task model training with LLM support.