Unsupervised Selective Rationalization with Noise Injection
Adam Storek Melanie Subbiah Kathleen McKeown
A major issue with using deep learning models in sensitive applications is
that they provide no explanation for their output. To address this problem,
unsupervised selective rationalization produces rationales alongside
predictions by chaining two jointly-trained components, a rationale generator
and a predictor. Although this architecture guarantees that the prediction
relies solely on the rationale, it does not ensure that the rationale contains
a plausible explanation for the prediction. We introduce a novel training
technique that effectively limits generation of implausible rationales by
injecting noise between the generator and the predictor. Furthermore, we
propose a new benchmark for evaluating unsupervised selective rationalization
models using movie reviews from existing datasets. We achieve sizeable
improvements in rationale plausibility and task accuracy over the
state-of-the-art across a variety of tasks, including our new benchmark, while
maintaining or improving model faithfulness.