基于多信息深度学习的3D点云语义分割

点云语义分割是三维点云数据处理的基础步骤,是三维场景理解分析、重建和目标识别的关键环节。针对目前对三维点云进行语义分割使用的点云信息少和精度不高的问题,本文在利用点云三维坐标信息的基础上,增加了点云RGB信息和所属房间的归一化坐标信息,从而丰富了神经网络输入端的信息量,进一步提高了模型的分割精度,最后利用PointNet++对改进后的三维点云语义分割效果进行检验,实验表明:在丰富了网络输入端的数据信息后,模型的总体准确度提高了6.65 %。     

基于语义信息补偿全局特征的物体点云分类分割

3D点云数据处理在物体分割、医学图像分割和虚拟现实等领域起到了重要作用。然而现有3D点云学习网络全局特征提取范围小,难以描述局部高级语义信息,进而导致点云特征表述不完整。针对这些问题,提出一种基于语义信息补偿全局特征的物体点云分类分割网络。首先,将输入的点云数据对齐到规范空间,进行数据的输入转换预处理。然后,利用扩张边缘卷积模块提取转换后数据的每一层特征,并叠加生成全局特征。而在局部特征提取时,利用提取到的低级语义信息来描述高级语义信息和有效几何特征,用于补偿全局特征中遗漏的点云特征。最后,融合全局特征和局部高级语义信息得到点云的整体特征。实验结果表明,文中方法在分类和分割性能上优于目前经典和新颖的算法。     

基于即时稠密三维重构的无人机视觉定位

传统景象匹配定位方法在用于低空无人机定位时,易因低空航拍图像视场小,且与卫星图像（带有地理信息）的拍摄角度差异大而失败.本文提出了一种基于即时稠密三维重构的无人机视觉定位方法,通过将稠密三维点云与卫星图像匹配以实现无人机定位.首先根据图像序列快速估计摄像机位姿,而后使用多深度图协同去噪与优化算法生成稠密三维点云,随后通过变换观察视角由稠密三维点云生成与卫星图像拍摄视角相近的虚拟视图,最后将虚拟视图与卫星图像匹配并得到无人机的地理坐标.由于稠密三维点云包含多张图像的信息,覆盖面积大,且可变化观察视角,因此能够有效克服上述两个问题.实验证明了本文方法的有效性.     

面向室内动态场景的VSLAM

传统VSLAM算法基于静态场景实现,其在室内动态场景下定位精度退化,三维稀疏点云地图也会出现动态特征点误匹配等问题.文中在ORB-SLAM2框架上进行改进,结合Mask R-CNN进行图像的语义分割,剔除位于动态物体上的动态特征点,优化了相机位姿,得到了静态的三维稀疏点云地图.在公开的TUM数据集上的实验结果表明,结合...     

Light bottle transformer based large scale point cloud classification

With the rapid development of computer vision, point clouds technique was widely used in practical applications, such as obstacle detection, roadside detection, smart city construction, etc. However, how to efficiently identify the large scale point clouds is still an open challenge. For relieving the large computation consumption and low accuracy problem in point cloud classification, a large scale point cloud classification framework based on light bottle transformer (light-BotNet) is proposed. Firstly, the two-dimensional (2D) and three-dimensional (3D) feature values of large scale point cloud were extracted for constructing point cloud feature images, which employed the prior knowledge to normalize the point cloud features. Then, the feature images are input to the classification network, and the light-BotNet network is applied for point cloud classification. It is an interesting attempt to combine the traditional image features with the transformer network. For proving the performance of the proposed method, the large scale point cloud benchmark Oakland 3D is utilized. In the experiments, the proposed method achieved 98.1% accuracy on the Oakland 3D dataset. Compared with the other methods, it can both reduce the memory consumption and improve the classification accuracy in large scale point cloud classification.     

Semantic segmentation of 3D point cloud based on contextual attention CNN

Aiming at the under-segmentation of 3D point cloud semantic segmentation caused by the lack of contextual fine-grained information of the point cloud,an algorithm based on contextual attention CNN was proposed for 3D point cloud semantic segmentation.Firstly,the fine-grained features in local area of the point cloud were mined through the attention coding mechanism.Secondly,the contextual features between multi-scale local areas were captured by the contextual recurrent neural network coding mechanism and compensated with the fine-grained local features.Finally,the multi-head mechanism was used to enhance the generalization ability of the network.Experiments show that the mIoU of the proposed algorithm on the three standard datasets of ShapeNet Parts,S3DIS and vKITTI are 85.4%,56.7% and 38.1% respectively,which has good segmentation performance and good generalization ability.     

点云编码综述

点云编码是支撑点云广泛应用的关键技术之一,是近期技术研究和标准化领域的热点。对点云几何信息和属性信息编码技术演进进行了回顾,并针对稠密点云和稀疏点云的几种典型编码方法的编码效率进行了比较。未来点云编码研究将集中于利用帧间预测去除动态点云的不同帧之间的相关性,以及端到端点云编码、任务驱动的点云编码等方面。     

顾及多尺度监督的点云语义分割

针对复杂场景点云分割精度不高、神经网络隐藏单元缺乏直接监督,难以提取语义明确的点云特征等问题,提出了一种将多尺度监督和SCF-Net相结合的点云语义分割网络。首先构建了一个类别信息生成模块,记录编码器中隐藏单元感受野内的类别,用于解码器中辅助分类器的监督学习。其次将解码阶段的点云类别预测任务分解成一系列点云感受野类别预测任务,通过对解码器中每一层添加辅助分类器,预测当前阶段点云感受野类别,编码阶段生成的类别信息作为标签监督网络学习。模型从粗到细地推理点云感受野类别,最终预测得到点云语义标签。实验结果表明,该方法能够有效提取点云关键信息,提高语义分割精度。     

自适应局部邻域特征点提取和匹配的点云配准

点云配准是三维重建的关键技术之一。针对点云匹配中迭代最近点算法(ICP)速率低、对初始位置要求高的问题,提出了一种基于自适应局部邻域特征点提取和匹配的点云配准方法。首先根据局部表面变化因子与平均变化因子的大小关系,自适应地提取特征点;其次利用快速点特征直方图（FPFH）综合描述每个特征点的局部信息,结合随机抽样一致性（RANSAC）算法实现粗配准;最后根据得到的初始变换矩阵和基于特征点的ICP算法实现精配准。对斯坦福数据集、含噪声的点云以及场景点云进行配准实验,实验结果表明:所提出的特征点提取算法能高效地提取点云的特征;相比于其他特征点检测方法,所提方法在粗配准中的配准精度和配准速度更高,且抗噪性能更好;与ICP算法相比,基于文中特征点的ICP算法在斯坦福数据集和场景点云中的配准速度提升了约10倍,在含噪声的点云中,能根据所提取的特征点高效地进行配准。该研究为提高三维重建和目标识别的匹配效率提供了一种高效的方法。     

基于体素化图卷积网络的三维点云目标检测方法

针对激光雷达点云的稀疏性和空间离散分布的特点,通过结合体素划分和图表示方法设计了新的图卷积特征提取模块,提出一种基于体素化图卷积神经网络的激光雷达三维点云目标检测算法。该方法通过消除传统3D卷积神经网络的计算冗余性,不仅提升了网络的目标检测能力,并且提高了点云拓扑信息的分析能力。文中设计的方法在KITTI公开数据集的车辆、行人、骑行者的3D目标检测和鸟瞰图目标检测任务的检测性能相比基准网络均有了有效提升,尤其在车辆3D目标检测任务上最高提升了13.75%。实验表明:该方法采用图卷积特征提取模块有效提高了网络整体检测性能和数据拓扑关系的学习能力,为三维点云目标检测任务提供了新的方法。     

改进贪婪投影三角化算法的激光点云快速三维重建

通过三维激光扫描仪获取的点云数据具有密度大、精度高等特点。本文针对贪婪投影三角化算法在对采集的大量点云数据进行三维重建时耗时长,重构的模型表面不够光滑,存在细小孔洞的问题,提出一种改进的点云三维重建算法。该方法首先用体像素网格滤波算法对点云进行下采样;然后使用移动最小二乘算法对输入的点云进行平滑及重采样,并且使用八叉树来代替KD树进行近邻域搜索;最后使用基于移动最小二乘算法的点云法线估计的贪婪投影三角化算法对点云进行重建。经过实验验证,该方法可以缩短重建时间,减少孔洞,并构建出平滑、点云拓扑结构更为准确的模型。     

3D dense reconstruction from 2D video sequence via 3D geometric segmentation

3D reconstruction is a major problem in computer vision. This paper considers the problem of reconstructing 3D structures, given a 2D video sequence. This problem is challenging since it is difficult to identify the trajectory of each object point/pixel over time. Traditional stereo 3D reconstruction methods and volumetric 3D reconstruction methods suffer from the blank wall problem, and the estimated dense depth map is not smooth, resulting in loss of actual geometric structures such as planes. To retain geometric structures embedded in the 3D scene, this paper proposes a novel surface fitting approach for 3D dense reconstruction. Specifically, we develop an expanded deterministic annealing algorithm to decompose 3D point cloud to multiple geometric structures, and estimate the parameters of each geometric structure. In this paper, we only consider plane structure, but our methodology can be extended to other parametric geometric structures such as spheres, cylinders, and cones. The experimental results show that the new approach is able to segment 3D point cloud into appropriate geometric structures and generate accurate 3D dense depth map.     

基于密集特征融合的无监督单目深度估计

针对无监督单目深度估计生成深度图质量低、边界模糊、伪影过多等问题,该文提出基于密集特征融合的深度网络编解码结构。设计密集特征融合层(DFFL)并将其以密集连接的形式填充U型编解码器,同时精简编码器部分,实现编、解码器的性能均衡。在训练过程中,将校正后的双目图像输入给网络,以重构视图的相似性约束网络生成视差图。测试时,根据已知的相机基线距离与焦距将生成的视差图转换为深度图。在KITTI数据集上的实验结果表明,该方法在预测精度和误差值上优于现有的算法。     

3D reconstruction and multiple point cloud registration using a low precision RGB-D sensor

A 3D reconstruction method using feature points is presented and the parameters used to improve the reconstruction are discussed. The precision of the 3D reconstruction is improved by combining point clouds obtained from different viewpoints using structured light. A well-known algorithm for point cloud registration is the ICP (Iterative Closest Point) that determines the rotation and translation that, when applied to one of the point clouds, places both point clouds optimally. The ICP algorithm iteratively executes two main steps: point correspondence determination and registration algorithm. The point correspondence determination is a module that, if not properly executed, can make the ICP converge to a local minimum. To overcome this drawback, two techniques were used. A meaningful set of 3D points using a technique known as SIFT (Scale-invariant feature transform) was obtained and an ICP that uses statistics to generate a dynamic distance and color threshold to the distance allowed between closest points was implemented. The reconstruction precision improvement was implemented using meaningful point clouds and the ICP to increase the number of points in the 3D space. The surface reconstruction is performed using marching cubes and filters to remove the noise and to smooth the surface. The factors that influence the 3D reconstruction precision are here discussed and analyzed. A detailed discussion of the number of frames used by the ICP and the ICP parameters is presented.     

Fine registration of 3D point clouds fusing structural and photometric information using an RGB-D camera

We address the problem of accurate and efficient alignment of 3D point clouds captured by an RGB-D (Kinect-style) camera from different viewpoints. While the Iterative Closest Point (ICP) algorithm has been widely used for dense point cloud matching, it is limited in its ability to produce accurate results in challenging scenarios involving objects that lack structural features and have significant camera view changes. In this paper, we introduce a new cost function with dynamic weights for the ICP algorithm to tackle this problem. It balances the significance of structural and photometric features with dynamically adjusted weights to improve the error minimization process. Our algorithm also includes a novel outlier rejection method, which adopts adaptive thresholding at each ICP iteration, using both the structural information of the object and the spatial distances of sparse SIFT feature pairs. The effectiveness of our proposed approach is demonstrated by experimental results from various challenging scenarios. We obtained superior registration accuracy than related previous methods, at the same time maintaining low computational requirements.     

AMSFF-Net: Attention-Based Multi-Stream Feature Fusion Network for Single Image Dehazing

In this paper, an end-to-end convolutional neural network is proposed to recover haze-free image named as Attention-Based Multi-Stream Feature Fusion Network (AMSFF-Net). The encoder-decoder network structure is used to construct the network. An encoder generates features at three resolution levels. The multi-stream features are extracted using residual dense blocks and fused by feature fusion blocks. AMSFF-Net has ability to pay more attention to informative features at different resolution levels using pixel attention mechanism. A sharp image can be recovered by the good kernel estimation. Further, AMSFF-Net has ability to capture semantic and sharp textural details from the extracted features and retain high-quality image from coarse-to-fine using mixed-convolution attention mechanism at decoder. The skip connections decrease the loss of image details from the larger receptive fields. Moreover, deep semantic loss function emphasizes more semantic information in deep features. Experimental findings prove that the proposed method outperforms in synthetic and real-world images.     

融合注意力门控机制的大场景点云语义分割

室外大场景激光点云语义分割已成为3D场景理解、环境感知的关键性技术,在自动驾驶、智能机器人和增强现实(AR)等领域应用广泛。然而大场景的激光点云具有多目标、几何结构复杂,不同地物尺度变化大等特点,使得在稀疏的小目标点云(例如行人、摩托车等)上的分割性能较低。针对上述问题,本文提出一种融合注意力门控机制的室外点云语义分割算法,设计由注意力机制和多尺度上下文特征融合组成的注意力门控单元,提高对激光点云细粒度特征的表达,降低随机降采样过程中点云几何结构特征丢失程度,从而增强了网络对弱小目标的特征获取能力;同时设计基于共享MLP的平均池化单元,进一步简化自注意力局部特征聚合模块,有效地加速网络收敛,能高效地实现大场景点云的语义分割。本文方法在自动驾驶场景室外激光点云数据集SemanticKITTI上的实验表明,与文献RandLA Net相比,收敛速度提升483,平均交并比(mIoU)由539提升至545,提高06,尤其是在小目标上交并比(IoU)均有明显提高,person类和motorcycle类的交并比分别提高08和54。     

基于改进特征的序列图像三维重建

近年来序列图像三维重建技术得到了重视,提出了很多算法.本文对Harris特征提取算法进行了改进,并在在序列图像中进行特征跟踪,在特征的稀疏重建和稠密重建后,使用估计样本一致性算法实现三维重建,提高了序列图像三维重建的精度.     

Super-resolution without dense flow

Super-resolution is a widely applied technique that improves the resolution of input images by software methods. Most conventional reconstruction-based super-resolution algorithms assume accurate dense optical flow fields between the input frames, and their performance degrades rapidly when the motion estimation result is not accurate enough. However, optical flow estimation is usually difficult, particularly when complicated motion is presented in real-world videos. In this paper, we explore a new way to solve this problem by using sparse feature point correspondences between the input images. The feature point correspondences, which are obtained by matching a set of feature points, are usually precise and much more robust than dense optical flow fields. This is because the feature points represent well-selected significant locations in the image, and performing matching on the feature point set is usually very accurate. In order to utilize the sparse correspondences in conventional super-resolution, we extract an adaptive support region with a reliable local flow field from each corresponding feature point pair. The normalized prior is also proposed to increase the visual consistency of the reconstructed result. Extensive experiments on real data were carried out, and results show that the proposed algorithm produces high-resolution images with better quality, particularly in the presence of large-scale or complicated motion fields.     

基于激光点云数据的三维运动图像重构技术北大核心CSCD

为了解决三维运动图重构时存在的重构测量距离与实际距离误差大、激光点云数据数量多和重构图像清晰度对比低的问题,提出了基于激光点云数据的三维运动图像重构技术,通过配准多帧激光点云数据,从中获取激光点云数据集,再采用平面拟合方法对激光点云数据集实行去噪处理,最后利用曲面重构法完成对曲面模型的拟合,实现三维运动图像重构。实验结果表明,通过对三维运动图像重构进行测量距离与实际距离的对比、激光点云数据数量的对比和测量图像与实际图像清晰度的对比测试,验证了三维运动图像重构技术的实用性高。