Small Language Models Learn Enhanced Reasoning Skills from Medical
  Textbooks
Hyunjae Kim Hyeon Hwang Jiwoo Lee Sihyeon Park Dain Kim Taewhoo Lee Chanwoong Yoon Jiwoong Sohn Donghee Choi Jaewoo Kang
While recent advancements in commercial large language models (LM) have shown
promising results in medical tasks, their closed-source nature poses
significant privacy and security concerns, hindering their widespread use in
the medical field. Despite efforts to create open-source models, their limited
parameters often result in insufficient multi-step reasoning capabilities
required for solving complex medical problems. To address this, we introduce
Meerkat-7B, a novel medical AI system with 7 billion parameters. Meerkat-7B was
trained using our new synthetic dataset consisting of high-quality
chain-of-thought reasoning paths sourced from 18 medical textbooks, along with
diverse instruction-following datasets. Our system achieved remarkable accuracy
across seven medical benchmarks, surpassing GPT-3.5 by 13.1%, as well as
outperforming the previous best 7B models such as MediTron-7B and BioMistral-7B
by 13.4% and 9.8%, respectively. Notably, it surpassed the passing threshold of
the United States Medical Licensing Examination (USMLE) for the first time for
a 7B-parameter model. Additionally, our system offered more detailed free-form
responses to clinical queries compared to existing 7B and 13B models,
approaching the performance level of GPT-3.5. This significantly narrows the
performance gap with large LMs, showcasing its effectiveness in addressing
complex medical challenges.