High accuracy and geometry-consistent confidence prediction network for multi-view stereo

Abstract Confidence prediction task attempts to infer the correctness of estimated depth hypotheseshich has gained popularity recently in stereo matching and boosts the accuracy of disparity estimation. However, less attention is paid on confidence prediction of multi-view stereo (MVS), where multi-view depth estimation is a key step for high-quality reconstruction. In this work, we propose a Geometry-consistent Confidence prediction Network (GeoConfNet), where the correctness of a depth hypothesis is accurately predicted via a deep neural network that explores both spatial coherence and cross-view consistency. The proposed deep network consists of a feature extraction module, a U-Net-based fusion module and a confidence refinement module. Furthermore, we demonstrate that truncated signed distance field (TSDF) is a powerful cross-view feature which can be an effective complement to spatial features, thereby remarkably boosting confidence prediction accuracy of MVS. Exhaustive experiments on a variety of MVS datasets as well as stereo matching datasets clearly demonstrate that our method achieves significantly better performance than state-of-the-art methods in terms of area under the curve (AUC).