Multi-model fusion for Aerial Vision and Dialog Navigation based on
  human attention aids
Xinyi Wang Xuan Cui Danxu Li Fang Liu Licheng Jiao
Drones have been widely used in many areas of our daily lives. It relieves
people of the burden of holding a controller all the time and makes drone
control easier to use for people with disabilities or occupied hands. However,
the control of aerial robots is more complicated compared to normal robots due
to factors such as uncontrollable height. Therefore, it is crucial to develop
an intelligent UAV that has the ability to talk to humans and follow natural
language commands. In this report, we present an aerial navigation task for the
2023 ICCV Conversation History. Based on the AVDN dataset containing more than
3k recorded navigation trajectories and asynchronous human-robot conversations,
we propose an effective method of fusion training of Human Attention Aided
Transformer model (HAA-Transformer) and Human Attention Aided LSTM (HAA-LSTM)
model, which achieves the prediction of the navigation routing points and human
attention. The method not only achieves high SR and SPL metrics, but also shows
a 7% improvement in GP metrics compared to the baseline model.