CompactifAI: Extreme Compression of Large Language Models using
  Quantum-Inspired Tensor Networks
Andrei Tomut Saeed S. Jahromi Sukhbinder Singh Faysal Ishtiaq Cesar Muñoz Prabdeep Singh Bajaj Ali Elborady Gianni del Bimbo Mehrazin Alizadeh David Montero Pablo Martin-Ramiro Muhammad Ibrahim Oussama Tahiri Alaoui John Malcolm Samuel Mugel Roman Orus
Large Language Models (LLMs) such as ChatGPT and LlaMA are advancing rapidly
in generative Artificial Intelligence (AI), but their immense size poses
significant challenges, such as huge training and inference costs, substantial
energy demands, and limitations for on-site deployment. Traditional compression
methods such as pruning, distillation, and low-rank approximation focus on
reducing the effective number of neurons in the network, while quantization
focuses on reducing the numerical precision of individual weights to reduce the
model size while keeping the number of neurons fixed. While these compression
methods have been relatively successful in practice, there's no compelling
reason to believe that truncating the number of neurons is an optimal strategy.
In this context, this paper introduces CompactifAI, an innovative LLM
compression approach using quantum-inspired Tensor Networks that focuses on the
model's correlation space instead, allowing for a more controlled, refined and
interpretable model compression. Our method is versatile and can be implemented
with - or on top of - other compression techniques. As a benchmark, we
demonstrate that CompactifAI alone enables compression of the LlaMA-2 7B model
to only $30\%$ of its original size while recovering over $90\%$ of the
original accuracy after a brief distributed retraining.