MP-SfM: Monocular Surface Priors for Robust Structure-from-Motion

While Structure-from-Motion (SfM) has seen much progress over the years, state-of-the-art systems are prone to failure when facing extreme viewpoint changes in low-overlap or low-parallax conditions.Because capturing images that avoid both pitfalls is challenging, this severely limits the wider use of SfM, especially by non-expert users.In this paper, we overcome both limitations by augmenting the classical SfM paradigm with monocular depth and normal priors, which can be inferred by deep neural networks with increasing accuracy.Our approach is significantly more robust than existing ones in extreme low- or high-overlap scenarios but retains state-of-the-art performance in easier, nominal conditions thanks to a tight integration of monocular and multi-view constraints.We also show that monocular priors can help reject faulty associations due to symmetries, which is a long-standing problem for SfM.Thanks to principled uncertainty propagation, our approach is robust to errors in the priors, can handle priors inferred by different models with little tuning, and will thus easily benefit from future progress in monocular depth and normal estimation.