PREADD: Prefix-Adaptive Decoding for Controlled Text Generation
Jonathan Pei Kevin Yang Dan Klein
We propose Prefix-Adaptive Decoding (PREADD), a flexible method for
controlled text generation. Unlike existing methods that use auxiliary expert
models to control for attributes, PREADD does not require an external model,
instead relying on linearly combining output logits from multiple prompts.
Specifically, PREADD contrasts the output logits generated using a raw prompt
against those generated using a prefix-prepended prompt, enabling both positive
and negative control with respect to any attribute encapsulated by the prefix.
We evaluate PREADD on three tasks -- toxic output mitigation, gender bias
reduction, and sentiment control -- and find that PREADD outperforms not only
prompting baselines, but also an auxiliary-expert control method, by 12% or
more in relative gain on our main metrics for each task.