Reviewer2: Optimizing Review Generation Through Prompt Generation
Zhaolin Gao Kianté Brantley Thorsten Joachims
Recent developments in LLMs offer new opportunities for assisting authors in
improving their work. In this paper, we envision a use case where authors can
receive LLM-generated reviews that uncover weak points in the current draft.
While initial methods for automated review generation already exist, these
methods tend to produce reviews that lack detail, and they do not cover the
range of opinions that human reviewers produce. To address this shortcoming, we
propose an efficient two-stage review generation framework called Reviewer2.
Unlike prior work, this approach explicitly models the distribution of possible
aspects that the review may address. We show that this leads to more detailed
reviews that better cover the range of aspects that human reviewers identify in
the draft. As part of the research, we generate a large-scale review dataset of
27k papers and 99k reviews that we annotate with aspect prompts, which we make
available as a resource for future research.