Automating Research Synthesis with Domain-Specific Large Language Model
  Fine-Tuning
Teo Susnjak Peter Hwang Napoleon H. Reyes Andre L. C. Barczak Timothy R. McIntosh Surangika Ranathunga
This research pioneers the use of fine-tuned Large Language Models (LLMs) to
automate Systematic Literature Reviews (SLRs), presenting a significant and
novel contribution in integrating AI to enhance academic research
methodologies. Our study employed the latest fine-tuning methodologies together
with open-sourced LLMs, and demonstrated a practical and efficient approach to
automating the final execution stages of an SLR process that involves knowledge
synthesis. The results maintained high fidelity in factual accuracy in LLM
responses, and were validated through the replication of an existing
PRISMA-conforming SLR. Our research proposed solutions for mitigating LLM
hallucination and proposed mechanisms for tracking LLM responses to their
sources of information, thus demonstrating how this approach can meet the
rigorous demands of scholarly research. The findings ultimately confirmed the
potential of fine-tuned LLMs in streamlining various labor-intensive processes
of conducting literature reviews. Given the potential of this approach and its
applicability across all research domains, this foundational study also
advocated for updating PRISMA reporting guidelines to incorporate AI-driven
processes, ensuring methodological transparency and reliability in future SLRs.
This study broadens the appeal of AI-enhanced tools across various academic and
research fields, setting a new standard for conducting comprehensive and
accurate literature reviews with more efficiency in the face of ever-increasing
volumes of academic studies.