Single-Stage 3D Geometry-Preserving Depth Estimation Model Training on
  Dataset Mixtures with Uncalibrated Stereo Data
Nikolay Patakin Mikhail Romanov Anna Vorontsova Mikhail Artemyev Anton Konushin
Nowadays, robotics, AR, and 3D modeling applications attract considerable
attention to single-view depth estimation (SVDE) as it allows estimating scene
geometry from a single RGB image. Recent works have demonstrated that the
accuracy of an SVDE method hugely depends on the diversity and volume of the
training data. However, RGB-D datasets obtained via depth capturing or 3D
reconstruction are typically small, synthetic datasets are not photorealistic
enough, and all these datasets lack diversity. The large-scale and diverse data
can be sourced from stereo images or stereo videos from the web. Typically
being uncalibrated, stereo data provides disparities up to unknown shift
(geometrically incomplete data), so stereo-trained SVDE methods cannot recover
3D geometry. It was recently shown that the distorted point clouds obtained
with a stereo-trained SVDE method can be corrected with additional point cloud
modules (PCM) separately trained on the geometrically complete data. On the
contrary, we propose GP$^{2}$, General-Purpose and Geometry-Preserving training
scheme, and show that conventional SVDE models can learn correct shifts
themselves without any post-processing, benefiting from using stereo data even
in the geometry-preserving setting. Through experiments on different dataset
mixtures, we prove that GP$^{2}$-trained models outperform methods relying on
PCM in both accuracy and speed, and report the state-of-the-art results in the
general-purpose geometry-preserving SVDE. Moreover, we show that SVDE models
can learn to predict geometrically correct depth even when geometrically
complete data comprises the minor part of the training set.