Camera Based mmWave Beam Prediction: Towards Multi-Candidate Real-World
  Scenarios
Gouranga Charan Muhammad Alrabeiah Tawfik Osman Ahmed Alkhateeb
Leveraging sensory information to aid the millimeter-wave (mmWave) and
sub-terahertz (sub-THz) beam selection process is attracting increasing
interest. This sensory data, captured for example by cameras at the
basestations, has the potential of significantly reducing the beam sweeping
overhead and enabling highly-mobile applications. The solutions developed so
far, however, have mainly considered single-candidate scenarios, i.e.,
scenarios with a single candidate user in the visual scene, and were evaluated
using synthetic datasets. To address these limitations, this paper extensively
investigates the sensing-aided beam prediction problem in a real-world
multi-object vehicle-to-infrastructure (V2I) scenario and presents a
comprehensive machine learning-based framework. In particular, this paper
proposes to utilize visual and positional data to predict the optimal beam
indices as an alternative to the conventional beam sweeping approaches. For
this, a novel user (transmitter) identification solution has been developed, a
key step in realizing sensing-aided multi-candidate and multi-user beam
prediction solutions. The proposed solutions are evaluated on the large-scale
real-world DeepSense $6$G dataset. Experimental results in realistic V2I
communication scenarios indicate that the proposed solutions achieve close to
$100\%$ top-5 beam prediction accuracy for the scenarios with single-user and
close to $95\%$ top-5 beam prediction accuracy for multi-candidate scenarios.
Furthermore, the proposed approach can identify the probable transmitting
candidate with more than $93\%$ accuracy across the different scenarios. This
highlights a promising approach for nearly eliminating the beam training
overhead in mmWave/THz communication systems.