Enhancing Inference Efficiency of Large Language Models: Investigating
  Optimization Strategies and Architectural Innovations
Georgy Tyukin
Large Language Models are growing in size, and we expect them to continue to
do so, as larger models train quicker. However, this increase in size will
severely impact inference costs. Therefore model compression is important, to
retain the performance of larger models, but with a reduced cost of running
them. In this thesis we explore the methods of model compression, and we
empirically demonstrate that the simple method of skipping latter attention
sublayers in Transformer LLMs is an effective method of model compression, as
these layers prove to be redundant, whilst also being incredibly
computationally expensive. We observed a 21% speed increase in one-token
generation for Llama 2 7B, whilst surprisingly and unexpectedly improving
performance over several common benchmarks.