Dynamic graph transformer for 3D object detection

LiDAR-based 3D detection is critical in autonomous driving perception systems. However, point-based 3D object detection that directly learns from point clouds is challenging owing to the sparsity and irregularity of LiDAR point clouds. Existing point-based methods are limited by fixed local relationships and the sparsity of distant and occluded objects. To address these issues, we propose a dynamic graph transformer 3D object detection network (DGT-Det3D) based on a dynamic graph transformer (DGT) module and a proposal-aware fusion (PAF) module. The DGT module is built on a dynamic graph and graph-aware self-attention module, which adaptively concentrates on the foreground points and encodes the graph to capture long-range dependencies. With the DGT module, DGT-Det3D has better capability to detect distant and occluded objects. To further refine the proposals, our PAF module fully integrates the proposal-aware spatial information and combines it with the point-wise semantic features from the first stage. Extensive experiments on the KITTI dataset demonstrate that our approach achieves state-of-the-art accuracy for point-based methods. In addition, DGT brings significant improvements when combined with state-of-the-art methods on the Waymo open dataset.