SummQA at MEDIQA-Chat 2023:In-Context Learning with GPT-4 for Medical
  Summarization
Yash Mathur Sanketh Rangreji Raghav Kapoor Medha Palavalli Amanda Bertsch Matthew R. Gormley
Medical dialogue summarization is challenging due to the unstructured nature
of medical conversations, the use of medical terminology in gold summaries, and
the need to identify key information across multiple symptom sets. We present a
novel system for the Dialogue2Note Medical Summarization tasks in the MEDIQA
2023 Shared Task. Our approach for section-wise summarization (Task A) is a
two-stage process of selecting semantically similar dialogues and using the
top-k similar dialogues as in-context examples for GPT-4. For full-note
summarization (Task B), we use a similar solution with k=1. We achieved 3rd
place in Task A (2nd among all teams), 4th place in Task B Division Wise
Summarization (2nd among all teams), 15th place in Task A Section Header
Classification (9th among all teams), and 8th place among all teams in Task B.
Our results highlight the effectiveness of few-shot prompting for this task,
though we also identify several weaknesses of prompting-based approaches. We
compare GPT-4 performance with several finetuned baselines. We find that GPT-4
summaries are more abstractive and shorter. We make our code publicly
available.