Vehicle Cameras Guide mmWave Beams: Approach and Real-World V2V
  Demonstration
Tawfik Osman Gouranga Charan Ahmed Alkhateeb
Accurately aligning millimeter-wave (mmWave) and terahertz (THz) narrow beams
is essential to satisfy reliability and high data rates of 5G and beyond
wireless communication systems. However, achieving this objective is difficult,
especially in vehicle-to-vehicle (V2V) communication scenarios, where both
transmitter and receiver are constantly mobile. Recently, additional sensing
modalities, such as visual sensors, have attracted significant interest due to
their capability to provide accurate information about the wireless
environment. To that end, in this paper, we develop a deep learning solution
for V2V scenarios to predict future beams using images from a 360 camera
attached to the vehicle. The developed solution is evaluated on a real-world
multi-modal mmWave V2V communication dataset comprising co-existing 360 camera
and mmWave beam training data. The proposed vision-aided solution achieves
$\approx 85\%$ top-5 beam prediction accuracy while significantly reducing the
beam training overhead. This highlights the potential of utilizing vision for
enabling highly-mobile V2V communications.