CodePrompt: Improving Source Code-Related Classification with Knowledge
  Features through Prompt Learning
Yong Ma Senlin Luo Yu-Ming Shang Yifei Zhang Zhengjun Li
Researchers have explored the potential of utilizing pre-trained language
models, such as CodeBERT, to improve source code-related tasks. Previous
studies have mainly relied on CodeBERT's text embedding capability and the
`[CLS]' sentence embedding information as semantic representations for
fine-tuning downstream source code-related tasks. However, these methods
require additional neural network layers to extract effective features,
resulting in higher computational costs. Furthermore, existing approaches have
not leveraged the rich knowledge contained in both source code and related
text, which can lead to lower accuracy. This paper presents a novel approach,
CodePrompt, which utilizes rich knowledge recalled from a pre-trained model by
prompt learning and an attention mechanism to improve source code-related
classification tasks. Our approach initially motivates the language model with
prompt information to retrieve abundant knowledge associated with the input as
representative features, thus avoiding the need for additional neural network
layers and reducing computational costs. Subsequently, we employ an attention
mechanism to aggregate multiple layers of related knowledge for each task as
final features to boost their accuracy. We conducted extensive experiments on
four downstream source code-related tasks to evaluate our approach and our
results demonstrate that CodePrompt achieves new state-of-the-art performance
on the accuracy metric while also exhibiting computation cost-saving
capabilities.