MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View
  Stereo
Chenjie Cao Xinlin Ren Yanwei Fu
Recent advancements in learning-based Multi-View Stereo (MVS) methods have
prominently featured transformer-based models with attention mechanisms.
However, existing approaches have not thoroughly investigated the profound
influence of transformers on different MVS modules, resulting in limited depth
estimation capabilities. In this paper, we introduce MVSFormer++, a method that
prudently maximizes the inherent characteristics of attention to enhance
various components of the MVS pipeline. Formally, our approach involves
infusing cross-view information into the pre-trained DINOv2 model to facilitate
MVS learning. Furthermore, we employ different attention mechanisms for the
feature encoder and cost volume regularization, focusing on feature and spatial
aggregations respectively. Additionally, we uncover that some design details
would substantially impact the performance of transformer modules in MVS,
including normalized 3D positional encoding, adaptive attention scaling, and
the position of layer normalization. Comprehensive experiments on DTU,
Tanks-and-Temples, BlendedMVS, and ETH3D validate the effectiveness of the
proposed method. Notably, MVSFormer++ achieves state-of-the-art performance on
the challenging DTU and Tanks-and-Temples benchmarks.