Do Generative Large Language Models need billions of parameters?
Sia Gholami Marwan Omar
This paper presents novel systems and methodologies for the development of
efficient large language models (LLMs). It explores the trade-offs between
model size, performance, and computational resources, with the aim of
maximizing the efficiency of these AI systems. The research explores novel
methods that allow different parts of the model to share parameters, reducing
the total number of unique parameters required. This approach ensures that the
model remains compact without sacrificing its ability to learn and represent
complex language structures. This study provides valuable insights and tools
for creating more efficient and effective LLMs, contributing to a more
sustainable and accessible future for AI language modeling.