Fusion of aerial images with mean shift-based upsampled elevation data for improved building block classification

Nowadays there is an increasing demand for detailed 3D modeling of buildings using elevation data such as those acquired from LiDAR airborne scanners. The various techniques that have been developed for this purpose typically perform segmentation into homogeneous regions followed by boundary extraction and are based on some combination of LiDAR data, digital maps, satellite images and aerial orthophotographs. In the present work, our dataset includes an aerial RGB orthophoto, a DSM and a DTM with spatial resolutions of 20cm, 1m and 2m respectively. Next, a normalized DSM (nDSM) is generated and fused with the optical data in order to increase its resolution to 20cm. The proposed methodology can be described as a two-step approach. First, a nearest neighbor interpolation is applied on the low resolution nDSM to obtain a low quality, ragged, elevation image. Next, we performed a mean shift-based discontinuity preserving smoothing on the fused data. The outcome is on the one hand a more homogeneous RGB image, with smoothed terrace coloring while at the same time preserving the optical edges and on the other hand an upsampled elevation data with considerable improvement regarding region filling and “straightness” of elevation discontinuities. Besides the apparent visual assessment of the increased accuracy of building boundaries, the effectiveness of the proposed method is demonstrated using the processed dataset as input to five supervised classification methods. The performance of each method is evaluated using a subset of the test area as ground truth. Comparisons with classification results obtained with the original data demonstrate that preprocessing the input dataset using the mean shift algorithm improves significantly the performance of all tested classifiers for building block extraction.