Abstract Syntax Tree for Programming Language Understanding and
  Representation: How Far Are We?
Weisong Sun Chunrong Fang Yun Miao Yudu You Mengzhe Yuan Yuchen Chen Quanjun Zhang An Guo Xiang Chen Yang Liu Zhenyu Chen
Programming language understanding and representation (a.k.a code
representation learning) has always been a hot and challenging task in software
engineering. It aims to apply deep learning techniques to produce numerical
representations of the source code features while preserving its semantics.
These representations can be used for facilitating subsequent code-related
tasks. The abstract syntax tree (AST), a fundamental code feature, illustrates
the syntactic information of the source code and has been widely used in code
representation learning. However, there is still a lack of systematic and
quantitative evaluation of how well AST-based code representation facilitates
subsequent code-related tasks. In this paper, we first conduct a comprehensive
empirical study to explore the effectiveness of the AST-based code
representation in facilitating follow-up code-related tasks. To do so, we
compare the performance of models trained with code token sequence (Token for
short) based code representation and AST-based code representation on three
popular types of code-related tasks. Surprisingly, the overall quantitative
statistical results demonstrate that models trained with AST-based code
representation consistently perform worse across all three tasks compared to
models trained with Token-based code representation. Our further quantitative
analysis reveals that models trained with AST-based code representation
outperform models trained with Token-based code representation in certain
subsets of samples across all three tasks. We also conduct comprehensive
experiments to evaluate and reveal the impact of the choice of AST
parsing/preprocessing/encoding methods on AST-based code representation and
subsequent code-related tasks. Our study provides future researchers with
detailed guidance on how to select solutions at each stage to fully exploit
AST.