Neural Models for Source Code Synthesis and Completion
Mitodru Niyogi
Natural language (NL) to code suggestion systems assist developers in
Integrated Development Environments (IDEs) by translating NL utterances into
compilable code snippet. The current approaches mainly involve hard-coded,
rule-based systems based on semantic parsing. These systems make heavy use of
hand-crafted rules that map patterns in NL or elements in its syntax parse tree
to various query constructs and can only work on a limited subset of NL with a
restricted NL syntax. These systems are unable to extract semantic information
from the coding intents of the developer, and often fail to infer types, names,
and the context of the source code to get accurate system-level code
suggestions. In this master thesis, we present sequence-to-sequence deep
learning models and training paradigms to map NL to general-purpose programming
languages that can assist users with suggestions of source code snippets, given
a NL intent, and also extend auto-completion functionality of the source code
to users while they are writing source code. The developed architecture
incorporates contextual awareness into neural models which generate source code
tokens directly instead of generating parse trees/abstract meaning
representations from the source code and converting them back to source code.
The proposed pretraining strategy and the data augmentation techniques improve
the performance of the proposed architecture. The proposed architecture has
been found to exceed the performance of a neural semantic parser, TranX, based
on the BLEU-4 metric by 10.82%. Thereafter, a finer analysis for the parsable
code translations from the NL intent for CoNaLA challenge was introduced. The
proposed system is bidirectional as it can be also used to generate NL code
documentation given source code. Lastly, a RoBERTa masked language model for
Python was proposed to extend the developed system for code completion.