Are Generative AI systems Capable of Supporting Information Needs of
  Patients?
Shreya Rajagopal Subhashis Hazarika Sookyung Kim Yan-ming Chiou Jae Ho Sohn Hari Subramonyam Shiwali Mohan
Patients managing a complex illness such as cancer face a complex information
challenge where they not only must learn about their illness but also how to
manage it. Close interaction with healthcare experts (radiologists,
oncologists) can improve patient learning and thereby, their disease outcome.
However, this approach is resource intensive and takes expert time away from
other critical tasks. Given the recent advancements in Generative AI models
aimed at improving the healthcare system, our work investigates whether and how
generative visual question answering systems can responsibly support patient
information needs in the context of radiology imaging data. We conducted a
formative need-finding study in which participants discussed chest computed
tomography (CT) scans and associated radiology reports of a fictitious close
relative with a cardiothoracic radiologist. Using thematic analysis of the
conversation between participants and medical experts, we identified commonly
occurring themes across interactions, including clarifying medical terminology,
locating the problems mentioned in the report in the scanned image,
understanding disease prognosis, discussing the next diagnostic steps, and
comparing treatment options. Based on these themes, we evaluated two
state-of-the-art generative visual language models against the radiologist's
responses. Our results reveal variability in the quality of responses generated
by the models across various themes. We highlight the importance of
patient-facing generative AI systems to accommodate a diverse range of
conversational themes, catering to the real-world informational needs of
patients.