From Wrong To Right: A Recursive Approach Towards Vision-Language
  Explanation
Jiaxin Ge Sanjay Subramanian Trevor Darrell Boyi Li
Addressing the challenge of adapting pre-trained vision-language models for
generating insightful explanations for visual reasoning tasks with limited
annotations, we present ReVisE: a $\textbf{Re}$cursive $\textbf{Vis}$ual
$\textbf{E}$xplanation algorithm. Our method iteratively computes visual
features (conditioned on the text input), an answer, and an explanation, to
improve the explanation quality step by step until the answer converges. We
find that this multi-step approach guides the model to correct its own answers
and outperforms single-step explanation generation. Furthermore, explanations
generated by ReVisE also serve as valuable annotations for few-shot
self-training. Our approach outperforms previous methods while utilizing merely
5% of the human-annotated explanations across 10 metrics, demonstrating up to a
4.2 and 1.3 increase in BLEU-1 score on the VCR and VQA-X datasets,
underscoring the efficacy and data-efficiency of our method.