Panoramic Image Mosaics Assisted by LiDAR in Vehicle System

Panoramic sensing system has attracted great attention in intelligent vehicle because of the wide range of perception areas. In order to make the spatial direction of each pixel more precise in panoramic image, we propose a panoramic image mosaics assisted by LiDAR. Firstly, a fusion system consisting of LiDAR and panoramic camera is built, and the related calibration equipment is elaborately designed for the system considering the sparse point cloud from 16-line LiDAR. Secondly, the center point of calibration plate is utilized to construct the geometry constraint for the extrinsic calibration of dual fisheye cameras, and the rotation matrix of calibration parameters makes the alignment of dual fisheye images. Thirdly, in order to offset calibration deviation and the distance between optical centers, the cos(x) smoothing function is introduced to eliminate the splicing marks in final panoramic image. Experimental results show that the proposed method can effectively be used in panoramic image mosaics, and the spatial direction of each pixel in the panoramic image is accurate.