WorldCoder, a Model-Based LLM Agent: Building World Models by Writing
  Code and Interacting with the Environment
Hao Tang Darren Key Kevin Ellis
We give a model-based agent that builds a Python program representing its
knowledge of the world based on its interactions with the environment. The
world model tries to explain its interactions, while also being optimistic
about what reward it can achieve. We do this by extending work on program
synthesis via LLMs. We study our agent on gridworlds, finding our approach is
more sample-efficient compared to deep RL, and more compute-efficient compared
to ReAct-style agents.