LangProp: A code optimization framework using Language Models applied to
  driving
Shu Ishida Gianluca Corrado George Fedoseev Hudson Yeo Lloyd Russell Jamie Shotton João F. Henriques Anthony Hu
LangProp is a framework for iteratively optimizing code generated by large
language models (LLMs) in a supervised/reinforcement learning setting. While
LLMs can generate sensible solutions zero-shot, the solutions are often
sub-optimal. Especially for code generation tasks, it is likely that the
initial code will fail on certain edge cases. LangProp automatically evaluates
the code performance on a dataset of input-output pairs, as well as catches any
exceptions, and feeds the results back to the LLM in the training loop, so that
the LLM can iteratively improve the code it generates. By adopting a metric-
and data-driven training paradigm for this code optimization procedure, one
could easily adapt findings from traditional machine learning techniques such
as imitation learning, DAgger, and reinforcement learning. We demonstrate the
first proof of concept of automated code optimization for autonomous driving in
CARLA, showing that LangProp can generate interpretable and transparent driving
policies that can be verified and improved in a metric- and data-driven way.
Our code will be open-sourced and is available at
https://github.com/shuishida/LangProp.