Language-enhanced RNR-Map: Querying Renderable Neural Radiance Field
  maps with natural language
Francesco Taioli Federico Cunico Federico Girella Riccardo Bologna Alessandro Farinelli Marco Cristani
We present Le-RNR-Map, a Language-enhanced Renderable Neural Radiance map for
Visual Navigation with natural language query prompts. The recently proposed
RNR-Map employs a grid structure comprising latent codes positioned at each
pixel. These latent codes, which are derived from image observation, enable: i)
image rendering given a camera pose, since they are converted to Neural
Radiance Field; ii) image navigation and localization with astonishing
accuracy. On top of this, we enhance RNR-Map with CLIP-based embedding latent
codes, allowing natural language search without additional label data. We
evaluate the effectiveness of this map in single and multi-object searches. We
also investigate its compatibility with a Large Language Model as an
"affordance query resolver". Code and videos are available at
https://intelligolabs.github.io/Le-RNR-Map/