Knowledge-Centric Templatic Views of Documents
Isabel Cachola Silviu Cucerzan Allen Herring Vuksan Mijovic Erik Oveson Sujay Kumar Jauhar
Authors seeking to communicate with broader audiences often compose their
ideas about the same underlying knowledge in different documents and formats --
for example, as slide decks, newsletters, reports, brochures, etc. Prior work
in document generation has generally considered the creation of each separate
format to be different a task, developing independent methods for generation
and evaluation. This approach is suboptimal for the advancement of AI-supported
content authoring from both research and application perspectives because it
leads to fragmented learning processes, redundancy in models and methods, and
disjointed evaluation. Thus, in our work, we consider each of these documents
to be templatic views of the same underlying knowledge, and we aim to unify the
generation and evaluation of these templatic views of documents. We begin by
introducing an LLM-powered method to extract the most important information
from an input document and represent this information in a structured format.
We show that this unified representation can be used to generate multiple
templatic views with no supervision and with very little guidance, improving
over strong baselines. We additionally introduce a unified evaluation method
that is template agnostic, and can be adapted to building document generators
for heterogeneous downstream applications. Finally, we conduct a human
evaluation, which shows that humans prefer 82% of the downstream documents
generated with our method. Furthermore, the newly proposed evaluation metric
correlates more highly with human judgement than prior metrics, while providing
a unified evaluation method.