LanGWM: Language Grounded World Model
Rudra P. K. Poudel Harit Pandya Chao Zhang Roberto Cipolla
Recent advances in deep reinforcement learning have showcased its potential
in tackling complex tasks. However, experiments on visual control tasks have
revealed that state-of-the-art reinforcement learning models struggle with
out-of-distribution generalization. Conversely, expressing higher-level
concepts and global contexts is relatively easy using language.
  Building upon recent success of the large language models, our main objective
is to improve the state abstraction technique in reinforcement learning by
leveraging language for robust action selection. Specifically, we focus on
learning language-grounded visual features to enhance the world model learning,
a model-based reinforcement learning technique.
  To enforce our hypothesis explicitly, we mask out the bounding boxes of a few
objects in the image observation and provide the text prompt as descriptions
for these masked objects. Subsequently, we predict the masked objects along
with the surrounding regions as pixel reconstruction, similar to the
transformer-based masked autoencoder approach.
  Our proposed LanGWM: Language Grounded World Model achieves state-of-the-art
performance in out-of-distribution test at the 100K interaction steps
benchmarks of iGibson point navigation tasks. Furthermore, our proposed
technique of explicit language-grounded visual representation learning has the
potential to improve models for human-robot interaction because our extracted
visual features are language grounded.