Depth Prediction From a Single Image with 3D Consistency

This paper addresses the problem of depth prediction from a single image. Existing methods based on deep convolutional neural networks learn depth by minimizing the I1 or l2 loss between predicted depth and ground truth depth. However, training a network with such a loss function often suffers from distortions in predicted 3 $D$ projections. In this paper, we propose a novel depth prediction method by effectively introducing 3D consistency. The proposed color consistency loss and 3D gradient consistency loss can enforce the network to learn depth with less distortions. Experiments on NYU Depth V2 dataset show that our depth predictions are competitive with state of the art methods and lead to faithful 3D projections.