TransTouch: Learning Transparent Objects Depth Sensing Through Sparse
  Touches
Liuyu Bian Pengyang Shi Weihang Chen Jing Xu Li Yi Rui Chen
Transparent objects are common in daily life. However, depth sensing for
transparent objects remains a challenging problem. While learning-based methods
can leverage shape priors to improve the sensing quality, the labor-intensive
data collection in the real world and the sim-to-real domain gap restrict these
methods' scalability. In this paper, we propose a method to finetune a stereo
network with sparse depth labels automatically collected using a probing system
with tactile feedback. We present a novel utility function to evaluate the
benefit of touches. By approximating and optimizing the utility function, we
can optimize the probing locations given a fixed touching budget to better
improve the network's performance on real objects. We further combine tactile
depth supervision with a confidence-based regularization to prevent
over-fitting during finetuning. To evaluate the effectiveness of our method, we
construct a real-world dataset including both diffuse and transparent objects.
Experimental results on this dataset show that our method can significantly
improve real-world depth sensing accuracy, especially for transparent objects.