On Compositionality and Improved Training of NADO
Sidi Lu Wenbo Zhao Chenyang Tao Arpit Gupta Shanchan Wu Tagyoung Chung Nanyun Peng
NeurAlly-Decomposed Oracle (NADO) is a powerful approach for controllable
generation with large language models. Differentiating from finetuning/prompt
tuning, it has the potential to avoid catastrophic forgetting of the large base
model and achieve guaranteed convergence to an entropy-maximized closed-form
solution without significantly limiting the model capacity. Despite its
success, several challenges arise when applying NADO to more complex scenarios.
First, the best practice of using NADO for the composition of multiple control
signals is under-explored. Second, vanilla NADO suffers from gradient vanishing
for low-probability control signals and is highly reliant on the
forward-consistency regularization. In this paper, we study the aforementioned
challenges when using NADO theoretically and empirically. We show we can
achieve guaranteed compositional generalization of NADO with a certain
practice, and propose a novel alternative parameterization of NADO to perfectly
guarantee the forward-consistency. We evaluate the improved training of NADO,
i.e. NADO++, on CommonGen. Results show that NADO++ improves the effectiveness
of the algorithm in multiple aspects.