Tabletop Transparent Scene Reconstruction via Epipolar-Guided Optical
  Flow with Monocular Depth Completion Prior
Xiaotong Chen Zheming Zhou Zhuo Deng Omid Ghasemalizadeh Min Sun Cheng-Hao Kuo Arnie Sen
Reconstructing transparent objects using affordable RGB-D cameras is a
persistent challenge in robotic perception due to inconsistent appearances
across views in the RGB domain and inaccurate depth readings in each
single-view. We introduce a two-stage pipeline for reconstructing transparent
objects tailored for mobile platforms. In the first stage, off-the-shelf
monocular object segmentation and depth completion networks are leveraged to
predict the depth of transparent objects, furnishing single-view shape prior.
Subsequently, we propose Epipolar-guided Optical Flow (EOF) to fuse several
single-view shape priors from the first stage to a cross-view consistent 3D
reconstruction given camera poses estimated from opaque part of the scene. Our
key innovation lies in EOF which employs boundary-sensitive sampling and
epipolar-line constraints into optical flow to accurately establish 2D
correspondences across multiple views on transparent objects. Quantitative
evaluations demonstrate that our pipeline significantly outperforms baseline
methods in 3D reconstruction quality, paving the way for more adept robotic
perception and interaction with transparent objects.