PACE: Improving Prompt with Actor-Critic Editing for Large Language
  Model
Yihong Dong Kangcheng Luo Xue Jiang Zhi Jin Ge Li
Large language models (LLMs) have showcased remarkable potential across
various tasks by conditioning on prompts. However, the quality of different
human-written prompts leads to substantial discrepancies in LLMs' performance,
and improving prompts usually necessitates considerable human effort and
expertise. To this end, this paper proposes Prompt with Actor-Critic Editing
(PACE) for LLMs to enable automatic prompt editing. Drawing inspiration from
the actor-critic algorithm in reinforcement learning, PACE leverages LLMs as
the dual roles of actors and critics, conceptualizing prompt as a type of
policy. PACE refines prompt, taking into account the feedback from both actors
performing prompt and critics criticizing response. This process helps LLMs
better align prompt to a specific task, thanks to real responses and thinking
from LLMs. We conduct extensive experiments on 24 instruction induction tasks
and 21 big-bench tasks. Experimental results indicate that PACE elevates the
relative performance of medium/low-quality human-written prompts by up to 98\%,
which has comparable performance to high-quality human-written prompts.
Moreover, PACE also exhibits notable efficacy for prompt generation.