Smoothly varying projective transformation for line segment matching

Abstract Line segment matching is important in applications that require recovering the 3D structure of objects (e.g., manmade objects in street-level scenarios). However, differentiating between true and false line matches is generally difficult without strong geometric constraints for line segments. Hence, additional constraints are forced to be used, sacrificing many true line matches. This study proposes a robust line segment matching method based on global projective transformation modeling. We develop a non-parametric motion regression formulation with a specially designed direct linear transformation-based cost function that reformulates the piecewise smoothly varying projective transformations as a global continuous model from highly noisy point matches. The resultant model can effectively approximate the real underlying image transformation and derive high-quality point matches. We apply the computed model and high-quality point matches to a point-correspondence-based line matching pipeline, which provides sufficient strict geometric constraints for first generating the pair-to-pair matches and then distilling the line-to-line matches. Extensive experiments conducted on two challenging line matching datasets show that the proposed method can obtain considerable correct line segment matches, outperforming the comparison methods in mean F-score by at least 15.5% on the benchmark dataset and 16.9% on the local dataset. Code is available at https://github.com/geovsion/SLEM .