DECIDER: A Rule-Controllable Decoding Strategy for Language Generation
  by Imitating Dual-System Cognitive Theory
Chen Xu Tian Lan Changlong Yu Wei Wang Jun Gao Yu Ji Qunxi Dong Kun Qian Piji Li Wei Bi Bin Hu
Lexicon-based constrained decoding approaches aim to control the meaning or
style of the generated text through certain target concepts. Existing
approaches over-focus the targets themselves, leading to a lack of high-level
reasoning about how to achieve them. However, human usually tackles tasks by
following certain rules that not only focuses on the targets but also on
semantically relevant concepts that induce the occurrence of targets. In this
work, we present DECIDER, a rule-controllable decoding strategy for constrained
language generation inspired by dual-system cognitive theory. Specifically, in
DECIDER, a pre-trained language model (PLM) is equiped with a logic reasoner
that takes high-level rules as input. Then, the DECIDER allows rule signals to
flow into the PLM at each decoding step. Extensive experimental results
demonstrate that DECIDER can effectively follow given rules to guide generation
direction toward the targets in a more human-like manner.