Fusion of urban 3D point clouds with thermal attributes using MLS data and TIR image sequences

Abstract Buildings take a large proportion of the total energy consumption in the city area in winter, therefore Thermal Infrared (TIR) images are widely used to evaluate the energy consumption and leakage of the building. To overcome the difficulties of image interpretation and occlusion in TIR images, utilizing thermal information in 3D structures that fuse TIR images with usable georeferenced 3D coordinates of buildings, is mandatory. To accomplish this mission, we propose a strategy to generate a thermal point cloud combining Mobile Laser Scanning (MLS) point clouds and TIR images. At first, a key-points extraction method based on line-intersection is presented, which helps to detect key-points from highly distorted TIR images. Then, the semi-automatic and the automatic correspondence determination algorithm based on restricted RANSAC for 6DOF pose estimation is proposed and tested. Finally, a non-local mean strategy for data fusion is applied, which integrates the thermal attributes from image sequences and the geometric property from the point clouds, resulting in a smooth representation of thermal point clouds with comprehensive radiance property for thermal analysis. The fusion result qualitatively presents thermal radiation of elements of buildings with detailed geometric information. The generated thermal point clouds contribute to evaluating the energy efficiency of buildings, or building blocks, analyzing the invisible building structure or pipelines for retrofit, and monitoring the energy usage of buildings for long-term development.