STRUM-LLM: Attributed and Structured Contrastive Summarization
Beliz Gunel James B. Wendt Jing Xie Yichao Zhou Nguyen Vo Zachary Fisher Sandeep Tata
Users often struggle with decision-making between two options (A vs B), as it
usually requires time-consuming research across multiple web pages. We propose
STRUM-LLM that addresses this challenge by generating attributed, structured,
and helpful contrastive summaries that highlight key differences between the
two options. STRUM-LLM identifies helpful contrast: the specific attributes
along which the two options differ significantly and which are most likely to
influence the user's decision. Our technique is domain-agnostic, and does not
require any human-labeled data or fixed attribute list as supervision.
STRUM-LLM attributes all extractions back to the input sources along with
textual evidence, and it does not have a limit on the length of input sources
that it can process. STRUM-LLM Distilled has 100x more throughput than the
models with comparable performance while being 10x smaller. In this paper, we
provide extensive evaluations for our method and lay out future directions for
our currently deployed system.