Multimodal Clinical Pseudo-notes for Emergency Department Prediction
  Tasks using Multiple Embedding Model for EHR (MEME)
Simon A. Lee Sujay Jain Alex Chen Arabdha Biswas Jennifer Fang Akos Rudas Jeffrey N. Chiang
In this work, we introduce Multiple Embedding Model for EHR (MEME), an
approach that views Electronic Health Records (EHR) as multimodal data. This
approach incorporates "pseudo-notes", textual representations of tabular EHR
concepts such as diagnoses and medications, and allows us to effectively employ
Large Language Models (LLMs) for EHR representation. This framework also adopts
a multimodal approach, embedding each EHR modality separately. We demonstrate
the effectiveness of MEME by applying it to several tasks within the Emergency
Department across multiple hospital systems. Our findings show that MEME
surpasses the performance of both single modality embedding methods and
traditional machine learning approaches. However, we also observe notable
limitations in generalizability across hospital institutions for all tested
models.