JAMDEC: Unsupervised Authorship Obfuscation using Constrained Decoding
  over Small Language Models
Jillian Fisher Ximing Lu Jaehun Jung Liwei Jiang Zaid Harchaoui Yejin Choi
The permanence of online content combined with the enhanced authorship
identification techniques calls for stronger computational methods to protect
the identity and privacy of online authorship when needed, e.g., blind reviews
for scientific papers, anonymous online reviews, or anonymous interactions in
the mental health forums. In this paper, we propose an unsupervised
inference-time approach to authorship obfuscation to address the unique
challenges of authorship obfuscation: lack of supervision data for diverse
authorship and domains, and the need for a sufficient level of revision beyond
simple paraphrasing to obfuscate the authorship, all the while preserving the
original content and fluency.
  We introduce JAMDEC, a user-controlled, inference-time algorithm for
authorship obfuscation that can be in principle applied to any text and
authorship. Our approach builds on small language models such as GPT2-XL in
order to help avoid disclosing the original content to proprietary LLM's APIs,
while also reducing the performance gap between small and large language models
via algorithmic enhancement. The key idea behind our approach is to boost the
creative power of smaller language models through constrained decoding, while
also allowing for user-specified controls and flexibility. Experimental results
demonstrate that our approach based on GPT2-XL outperforms previous
state-of-the-art methods based on comparably small models, while performing
competitively against GPT3.5 175B, a propriety model that is two orders of
magnitudes larger.