Blur aware metric depth estimation with multi-focus plenoptic cameras
Mathieu Labussière Céline Teulière Omar Ait-Aider
While a traditional camera only captures one point of view of a scene, a
plenoptic or light-field camera, is able to capture spatial and angular
information in a single snapshot, enabling depth estimation from a single
acquisition. In this paper, we present a new metric depth estimation algorithm
using only raw images from a multi-focus plenoptic camera. The proposed
approach is especially suited for the multi-focus configuration where several
micro-lenses with different focal lengths are used. The main goal of our blur
aware depth estimation (BLADE) approach is to improve disparity estimation for
defocus stereo images by integrating both correspondence and defocus cues. We
thus leverage blur information where it was previously considered a drawback.
We explicitly derive an inverse projection model including the defocus blur
providing depth estimates up to a scale factor. A method to calibrate the
inverse model is then proposed. We thus take into account depth scaling to
achieve precise and accurate metric depth estimates. Our results show that
introducing defocus cues improves the depth estimation. We demonstrate the
effectiveness of our framework and depth scaling calibration on relative depth
estimation setups and on real-world 3D complex scenes with ground truth
acquired with a 3D lidar scanner.