DictLLM: Harnessing Key-Value Data Structures with Large Language Models
  for Enhanced Medical Diagnostics
YiQiu Guo Yuchen Yang Ya Zhang Yu Wang Yanfeng Wang
Structured data offers a sophisticated mechanism for the organization of
information. Existing methodologies for the text-serialization of structured
data in the context of large language models fail to adequately address the
heterogeneity inherent in key-value structured data. These methods are not
ideal and frequently result in larger input sizes and poor adaptability to
input changes. In this paper, we introduce DictLLM, an innovative framework
designed to improve the modeling of key-value structured data, like medical
laboratory reports, for generating medical diagnoses. DictLLM integrates three
key components: (1) group positional encoding to maintain permutation
invariance, (2) hierarchical attention bias to capture the inherent bias in
structured data, and (3) an optimal transport alignment layer that aligns the
embedding generated by the dictionary encoder with the LLM, thereby producing a
sequence of fixed-length virtual tokens. We carry out experiments using various
LLM models on a comprehensive real-world medical laboratory report dataset for
automatic diagnosis generation, our findings illustrate that DictLLM
significantly outperforms established baseline methods and few-shot GPT-4
implementations in terms of both Rouge-L and Knowledge F1 scores. Furthermore,
our evaluation of the framework's scalability and robustness, through a series
of experiments, underscores its exceptional capability in accurately modeling
the complex key-value data structure of medical dictionary data.