Beyond the Chat: Executable and Verifiable Text-Editing with LLMs
Philippe Laban Jesse Vig Marti A. Hearst Caiming Xiong Chien-Sheng Wu
Conversational interfaces powered by Large Language Models (LLMs) have
recently become a popular way to obtain feedback during document editing.
However, standard chat-based conversational interfaces do not support
transparency and verifiability of the editing changes that they suggest. To
give the author more agency when editing with an LLM, we present InkSync, an
editing interface that suggests executable edits directly within the document
being edited. Because LLMs are known to introduce factual errors, Inksync also
supports a 3-stage approach to mitigate this risk: Warn authors when a
suggested edit introduces new information, help authors Verify the new
information's accuracy through external search, and allow an auditor to perform
an a-posteriori verification by Auditing the document via a trace of all
auto-generated content. Two usability studies confirm the effectiveness of
InkSync's components when compared to standard LLM-based chat interfaces,
leading to more accurate, more efficient editing, and improved user experience.