Distributed Radiance Fields for Edge Video Compression and Metaverse
  Integration in Autonomous Driving
Eugen Šlapak Matúš Dopiriak Mohammad Abdullah Al Faruque Juraj Gazda Marco Levorato
The metaverse is a virtual space that combines physical and digital elements,
creating immersive and connected digital worlds. For autonomous mobility, it
enables new possibilities with edge computing and digital twins (DTs) that
offer virtual prototyping, prediction, and more. DTs can be created with 3D
scene reconstruction methods that capture the real world's geometry,
appearance, and dynamics. However, sending data for real-time DT updates in the
metaverse, such as camera images and videos from connected autonomous vehicles
(CAVs) to edge servers, can increase network congestion, costs, and latency,
affecting metaverse services. Herein, a new method is proposed based on
distributed radiance fields (RFs), multi-access edge computing (MEC) network
for video compression and metaverse DT updates. RF-based encoder and decoder
are used to create and restore representations of camera images. The method is
evaluated on a dataset of camera images from the CARLA simulator. Data savings
of up to 80% were achieved for H.264 I-frame - P-frame pairs by using RFs
instead of I-frames, while maintaining high peak signal-to-noise ratio (PSNR)
and structural similarity index measure (SSIM) qualitative metrics for the
reconstructed images. Possible uses and challenges for the metaverse and
autonomous mobility are also discussed.