CoverUp: Coverage-Guided LLM-Based Test Generation
Juan Altmayer Pizzorno Emery D. Berger
This paper presents CoverUp, a novel system that drives the generation of
high-coverage Python regression tests via a combination of coverage analysis
and large-language models (LLMs). CoverUp iteratively improves coverage,
interleaving coverage analysis with dialogs with the LLM to focus its attention
on as yet uncovered lines and branches. The resulting test suites significantly
improve coverage over the current state of the art: compared to CodaMosa, a
hybrid LLM / search-based software testing system, CoverUp substantially
improves coverage across the board. On a per-module basis, CoverUp achieves
median line coverage of 81% (vs. 62%), branch coverage of 53% (vs. 35%) and
line+branch coverage of 78% (vs. 55%). We show that CoverUp's iterative,
coverage-guided approach is crucial to its effectiveness, contributing to
nearly half of its successes.