Emergent communication for AR
Ruxiao Chen Shuaishuai Guo
Mobile augmented reality (MAR) is widely acknowledged as one of the
ubiquitous interfaces to the digital twin and Metaverse, demanding unparalleled
levels of latency, computational power, and energy efficiency. The existing
solutions for realizing MAR combine multiple technologies like edge, cloud
computing, and fifth-generation (5G) networks. However, the inherent
communication latency of visual data imposes apparent limitations on the
quality of experience (QoE). To address the challenge, we propose an emergent
semantic communication framework to learn the communication protocols in MAR.
Specifically, we train two agents through a modified Lewis signaling game to
emerge a discrete communication protocol spontaneously. Based on this protocol,
two agents can communicate about the abstract idea of visual data through
messages with extremely small data sizes in a noisy channel, which leads to
message errors. To better simulate real-world scenarios, we incorporate channel
uncertainty into our training process. Experiments have shown that the proposed
scheme has better generalization on unseen objects than traditional object
recognition used in MAR and can effectively enhance communication efficiency
through the utilization of small-size messages.