Large Generative Model Assisted 3D Semantic Communication
Feibo Jiang Yubo Peng Li Dong Kezhi Wang Kun Yang Cunhua Pan Xiaohu You
Semantic Communication (SC) is a novel paradigm for data transmission in 6G.
However, there are several challenges posed when performing SC in 3D scenarios:
1) 3D semantic extraction; 2) Latent semantic redundancy; and 3) Uncertain
channel estimation. To address these issues, we propose a Generative AI Model
assisted 3D SC (GAM-3DSC) system. Firstly, we introduce a 3D Semantic Extractor
(3DSE), which employs generative AI models, including Segment Anything Model
(SAM) and Neural Radiance Field (NeRF), to extract key semantics from a 3D
scenario based on user requirements. The extracted 3D semantics are represented
as multi-perspective images of the goal-oriented 3D object. Then, we present an
Adaptive Semantic Compression Model (ASCM) for encoding these multi-perspective
images, in which we use a semantic encoder with two output heads to perform
semantic encoding and mask redundant semantics in the latent semantic space,
respectively. Next, we design a conditional Generative adversarial network and
Diffusion model aided-Channel Estimation (GDCE) to estimate and refine the
Channel State Information (CSI) of physical channels. Finally, simulation
results demonstrate the advantages of the proposed GAM-3DSC system in
effectively transmitting the goal-oriented 3D scenario.