伪点云修正增强激光雷达数据

目的 激光雷达在自动驾驶中具有重要意义，但其价格昂贵，且产生的激光线束数量仍然较少，造成采集的点云密度较稀疏。为了更好地感知周围环境，本文提出一种激光雷达数据增强算法，由双目图像生成伪点云并对伪点云进行坐标修正，进而实现激光雷达点云的稠密化处理，提高3D目标检测精度。此算法不针对特定的3D目标检测网络结构，是一种通用的点云稠密化方法。方法 首先利用双目RGB图像生成深度图像，根据先验的相机参数和深度信息计算出每个像素点在雷达坐标系下的粗略3维坐标，即伪点云。为了更好地分割地面，本文提出了循环RANSAC （random sample consensus）算法，引入了一个分离平面型非地面点云的暂存器，改进复杂场景下的地面分割效果。然后将原始点云进行地面分割后插入KDTree （k-dimensional tree），以伪点云中的每个点为中心在KDTree中搜索若干近邻点，基于这些近邻点进行曲面重建。根据曲面重建结果，设计一种计算几何方法导出伪点云修正后的精确坐标。最后，将修正后的伪点云与原始激光雷达点云融合得到稠密化点云。结果 实验结果表明，稠密化的点云在视觉上具有较好的质量，物体具有更加完整的形状和轮廓，并且在KITTI （Karlsruhe Institute of Technology and Toyota Technological Institute）数据集上提升了3D目标检测精度。在使用该数据增强方法后，KITTI数据集下AVOD （aggregate view object detection）检测方法的AP_3D-Easy （average precision of 3D object detection on easy setting）提升了8.25%，AVOD-FPN （aggregate view object detection with feature pyramid network）检测方法的AP_BEV-Hard （average precision of bird’s eye view on hard setting）提升了7.14%。结论 本文提出的激光雷达数据增强算法，实现了点云的稠密化处理，并使3D目标检测结果更加精确。

LiDAR data enhancement via pseudo-LiDAR point cloud correction

Objective Light detection and ranging (LiDAR) plays an important role in autonomous driving. Even if it is expensive, the number of equipped laser beams is still small, which results in a sparse point cloud density. The sparsity of the LiDAR point cloud makes 3D object detection difficult. The camera is another vital sensor used in autonomous driving because of the mature image recognition methods and its competitive price compared with LiDAR. However, it does not perform as well as LiDAR in 3D object detection task. To perceive the surrounding environment better, this study proposes a LiDAR data enhancement algorithm based on pseudo-LiDAR point cloud correction method to increase the density of LiDAR point cloud, thereby improving the accuracy of 3D object detection. This method has a wide application prospect because it is a general method to densify the point cloud and improve the detection accuracy, which does not depend on specific 3D object detection network structures. Method The algorithm can be divided into four steps. In the first step, the depth map is generated on the basis of the stereo RGB images by using depth estimation methods, such as pyramid stereo matching network (PSMNet) and DeepPruner. The approximate 3D coordinates of each pixel corresponding to the LiDAR coordinate system are calculated according to the camera parameters and depth information. The point cloud formed by approximate 3D coordinates is usually called pseudo-LiDAR point cloud. In the second step, the ground points in the original point cloud are removed by ground segmentation method because they will disturb the surface reconstruction process in the following step. In order to improve the performance of ground segmentation, this study designs an iterative random sample consensus (RANSAC) algorithm. A register is used in the iterative RANSAC algorithm to store the points extracted in each iteration, which are planar point cloud but not ground point cloud. The register ensures that the next iteration of RANSAC will not be affected by these non-ground planar points. The iterative RANSAC algorithm performs better than the normal RANSAC algorithm in complex scenarios where the number of points in the non-ground plane is larger than that in a ground plane or there exist multiple ground planes with different angles of inclination. In the third step, the original point cloud is inserted into a k-dimensional tree (KDTree) after ground segmentation. Thereafter, several neighboring points of each point in the pseudo-LiDAR point cloud are obtained by searching in the KDTree. On the basis of these neighboring points, surface reconstruction is performed to catch the local feature around the pseudo-LiDAR point. The Delaunay triangulation surface reconstruction method, which is considered as the optimal surface reconstruction algorithm by many researchers, is used in this step. The result of surface reconstruction is represented as several tiny triangles that cover the whole surface in 3D space. If the distance between the current and last processed approximate 3D point is within the KDTree searching radius, the surface reconstruction result of the last processed point is directly regarded as the surface reconstruction result of the current point to save time by skipping the KDTree searching and surface reconstruction process. Furthermore, because the process of KDTree searching and surface reconstruction is independent for each pseudo-LiDAR point, parallel computation based on OpenMP is used to speed up this step. In the fourth step, the precise 3D coordinates of pseudo-LiDAR point cloud corresponding to the LiDAR coordinate system are derived by the designed computational geometry method. In the computational geometry method, two different depth values are set for each pixel to obtain two points in the 3D coordinate system. The two points determine the line of the light path of this pixel. Then, the precise 3D coordinate of this pixel is considered as the closest point from the origin among all of the intersection points of the light path line and reconstruction triangle surfaces. This computational geometry method realizes the function of pseudo-LiDAR point coordinate correction. To prevent the loss of accuracy, the method avoids using division in the calculation process based on inequality analysis. Finally, the precise 3D points generated in step four are merged with the origin point cloud scanned by LiDAR to obtain the dense point cloud. Result After the densification, the objects in the point cloud have more complete shapes and contours than before, which means that the characteristics of objects are more obvious. In order to further verify the validity of this data enhancement method, the aggregate view object detection (AVOD) and aggregate view object detection with feature pyramid network (AVOD-FPN) methods are implemented to check whether the average precision of 3D object detection on the dense point cloud is higher than that of the original point cloud. The data enhancement algorithm was used on KITTI (Karlsruhe Institute of Technology and Toyota Technological Institute) dataset to obtain a dataset with dense LiDAR point clouds. Then, 3D object detection methods were implemented on the original LiDAR point cloud and dense LiDAR point cloud. After using this data enhancement method, the AP_3D-Easy of AVOD increased by 8.25%, and the AP_BEV-Hard of AVOD-FPN increased by 7.14%. Conclusion A vision-based LiDAR data enhancement algorithm was proposed to increase the density of LiDAR point cloud, thereby improving the accuracy of 3D object detection. The experimental results show that the dense point cloud has good visual quality, and the data enhancement method improves the accuracy of 3D object detection on KITTI dataset.