Leveraging Reinforcement Learning and Large Language Models for Code
  Optimization
Shukai Duan Nikos Kanakaris Xiongye Xiao Heng Ping Chenyu Zhou Nesreen K. Ahmed Guixiang Ma Mihai Capota Theodore L. Willke Shahin Nazarian Paul Bogdan
Code optimization is a daunting task that requires a significant level of
expertise from experienced programmers. This level of expertise is not
sufficient when compared to the rapid development of new hardware
architectures. Towards advancing the whole code optimization process, recent
approaches rely on machine learning and artificial intelligence techniques.
This paper introduces a new framework to decrease the complexity of code
optimization. The proposed framework builds on large language models (LLMs) and
reinforcement learning (RL) and enables LLMs to receive feedback from their
environment (i.e., unit tests) during the fine-tuning process. We compare our
framework with existing state-of-the-art models and show that it is more
efficient with respect to speed and computational usage, as a result of the
decrement in training steps and its applicability to models with fewer
parameters. Additionally, our framework reduces the possibility of logical and
syntactical errors. Toward evaluating our approach, we run several experiments
on the PIE dataset using a CodeT5 language model and RRHF, a new reinforcement
learning algorithm. We adopt a variety of evaluation metrics with regards to
optimization quality, and speedup. The evaluation results demonstrate that the
proposed framework has similar results in comparison with existing models using
shorter training times and smaller pre-trained models. In particular, we
accomplish an increase of 5.6% and 2.2 over the baseline models concerning the
%OP T and SP metrics.