Semi-Supervised Semantic Depth Estimation using Symbiotic Transformer
  and NearFarMix Augmentation
Md Awsafur Rahman Shaikh Anowarul Fattah
In computer vision, depth estimation is crucial for domains like robotics,
autonomous vehicles, augmented reality, and virtual reality. Integrating
semantics with depth enhances scene understanding through reciprocal
information sharing. However, the scarcity of semantic information in datasets
poses challenges. Existing convolutional approaches with limited local
receptive fields hinder the full utilization of the symbiotic potential between
depth and semantics. This paper introduces a dataset-invariant semi-supervised
strategy to address the scarcity of semantic information. It proposes the Depth
Semantics Symbiosis module, leveraging the Symbiotic Transformer for achieving
comprehensive mutual awareness by information exchange within both local and
global contexts. Additionally, a novel augmentation, NearFarMix is introduced
to combat overfitting and compensate both depth-semantic tasks by strategically
merging regions from two images, generating diverse and structurally consistent
samples with enhanced control. Extensive experiments on NYU-Depth-V2 and KITTI
datasets demonstrate the superiority of our proposed techniques in indoor and
outdoor environments.