Improving Open-Ended Text Generation via Adaptive Decoding
Wenhong Zhu Hongkun Hao Zhiwei He Yiming Ai Rui Wang
Current language models decode text token by token according to probabilistic
distribution, and determining the appropriate candidates for the next token is
crucial to ensure generation quality. This study introduces adaptive decoding,
a mechanism that empowers the language models to ascertain a sensible candidate
set during the generation process dynamically. Specifically, we introduce an
entropy-based metric called confidence and conceptualize determining the
optimal candidate set as a confidence-increasing process. The rationality of
including a token in the candidate set is assessed by leveraging the increment
of confidence, enabling the model to determine the most suitable candidate set
adaptively. The experimental results reveal that our method achieves higher
MAUVE and diversity in story generation tasks and maintains certain coherence,
underscoring its superiority over existing algorithms. The code is available at
https://github.com/zwhong714/adaptive_decoding.