End-to-end Story Plot Generator
Hanlin Zhu Andrew Cohen Danqing Wang Kevin Yang Xiaomeng Yang Jiantao Jiao Yuandong Tian
Story plots, while short, carry most of the essential information of a full
story that may contain tens of thousands of words. We study the problem of
automatic generation of story plots, which includes story premise, character
descriptions, plot outlines, etc. To generate a single engaging plot, existing
plot generators (e.g., DOC (Yang et al., 2022a)) require hundreds to thousands
of calls to LLMs (e.g., OpenAI API) in the planning stage of the story plot,
which is costly and takes at least several minutes. Moreover, the hard-wired
nature of the method makes the pipeline non-differentiable, blocking fast
specialization and personalization of the plot generator. In this paper, we
propose three models, $\texttt{OpenPlot}$, $\texttt{E2EPlot}$ and
$\texttt{RLPlot}$, to address these challenges. $\texttt{OpenPlot}$ replaces
expensive OpenAI API calls with LLaMA2 (Touvron et al., 2023) calls via careful
prompt designs, which leads to inexpensive generation of high-quality training
datasets of story plots. We then train an end-to-end story plot generator,
$\texttt{E2EPlot}$, by supervised fine-tuning (SFT) using approximately 13000
story plots generated by $\texttt{OpenPlot}$. $\texttt{E2EPlot}$ generates
story plots of comparable quality to $\texttt{OpenPlot}$, and is > 10$\times$
faster (1k tokens in only 30 seconds on average). Finally, we obtain
$\texttt{RLPlot}$ that is further fine-tuned with RLHF on several different
reward models for different aspects of story quality, which yields 60.0$\%$
winning rate against $\texttt{E2EPlot}$ along the aspect of suspense and
surprise.