Controllable Chest X-Ray Report Generation from Longitudinal
  Representations
Francesco Dalla Serra Chaoyang Wang Fani Deligianni Jeffrey Dalton Alison Q O'Neil
Radiology reports are detailed text descriptions of the content of medical
scans. Each report describes the presence/absence and location of relevant
clinical findings, commonly including comparison with prior exams of the same
patient to describe how they evolved. Radiology reporting is a time-consuming
process, and scan results are often subject to delays. One strategy to speed up
reporting is to integrate automated reporting systems, however clinical
deployment requires high accuracy and interpretability. Previous approaches to
automated radiology reporting generally do not provide the prior study as
input, precluding comparison which is required for clinical accuracy in some
types of scans, and offer only unreliable methods of interpretability.
Therefore, leveraging an existing visual input format of anatomical tokens, we
introduce two novel aspects: (1) longitudinal representation learning -- we
input the prior scan as an additional input, proposing a method to align,
concatenate and fuse the current and prior visual information into a joint
longitudinal representation which can be provided to the multimodal report
generation model; (2) sentence-anatomy dropout -- a training strategy for
controllability in which the report generator model is trained to predict only
sentences from the original report which correspond to the subset of anatomical
regions given as input. We show through in-depth experiments on the MIMIC-CXR
dataset how the proposed approach achieves state-of-the-art results while
enabling anatomy-wise controllable report generation.