Most Language Models can be Poets too: An AI Writing Assistant and
  Constrained Text Generation Studio
Allen Roush Sanjay Basu Akshay Moorthy Dmitry Dubovoy
Despite rapid advancement in the field of Constrained Natural Language
Generation, little time has been spent on exploring the potential of language
models which have had their vocabularies lexically, semantically, and/or
phonetically constrained. We find that most language models generate compelling
text even under significant constraints. We present a simple and universally
applicable technique for modifying the output of a language model by
compositionally applying filter functions to the language models vocabulary
before a unit of text is generated. This approach is plug-and-play and requires
no modification to the model. To showcase the value of this technique, we
present an easy to use AI writing assistant called Constrained Text Generation
Studio (CTGS). CTGS allows users to generate or choose from text with any
combination of a wide variety of constraints, such as banning a particular
letter, forcing the generated words to have a certain number of syllables,
and/or forcing the words to be partial anagrams of another word. We introduce a
novel dataset of prose that omits the letter e. We show that our method results
in strictly superior performance compared to fine-tuning alone on this dataset.
We also present a Huggingface space web-app presenting this technique called
Gadsby. The code is available to the public here:
https://github.com/Hellisotherpeople/Constrained-Text-Generation-Studio