Aligning LLMs for FL-free Program Repair
Junjielong Xu Ying Fu Shin Hwei Tan Pinjia He
Large language models (LLMs) have achieved decent results on automated
program repair (APR). However, the next token prediction training objective of
decoder-only LLMs (e.g., GPT-4) is misaligned with the masked span prediction
objective of current infilling-style methods, which impedes LLMs from fully
leveraging pre-trained knowledge for program repair. In addition, while some
LLMs are capable of locating and repairing bugs end-to-end when using the
related artifacts (e.g., test cases) as input, existing methods regard them as
separate tasks and ask LLMs to generate patches at fixed locations. This
restriction hinders LLMs from exploring potential patches beyond the given
locations.
  In this paper, we investigate a new approach to adapt LLMs to program repair.
Our core insight is that LLM's APR capability can be greatly improved by simply
aligning the output to their training objective and allowing them to refine the
whole program without first performing fault localization. Based on this
insight, we designed D4C, a straightforward prompting framework for APR. D4C
can repair 180 bugs correctly in Defects4J, with each patch being sampled only
10 times. This surpasses the SOTA APR methods with perfect fault localization
by 10% and reduces the patch sampling number by 90%. Our findings reveal that
(1) objective alignment is crucial for fully exploiting LLM's pre-trained
capability, and (2) replacing the traditional localize-then-repair workflow
with direct debugging is more effective for LLM-based APR methods. Thus, we
believe this paper introduces a new mindset for harnessing LLMs in APR.