基于长序列图像的三维重建技术研究*

提出一种基于长序列未定标图像的三维重建方法,并将其成功地应用于增强现实实例中.首先,基于传统KLT跟踪算法提出了一种针对序列图像的改进特征点匹配策略,通过特征点运动向量的预测减小了相应特征点的搜索范围,进一步根据相近特征点邻域窗口在透视畸变上的相似性大大提高了匹配算法的效率;在得到序列图像的匹配结果后,将传统基于仿射成像模型的测量矩阵(Measurement Matrix)保秩分解算法扩展到透视成像模型中,从而一次性得到整个场景的射影重建;进一步在摄像机自定标的基础上得到整个场景的三维欧氏模型和摄像机的成像矩阵.最后给出真实图像序列的三维重建实验结果,并成功地将其应用到增强现实实例中.     

结合三维几何形状信息和二维纹理的3D目标匹配

为了实现场景中三维目标与模型之间的匹配,提出了一种结合三维几何形状信息和二维纹理的三维目标匹配方法。首先提取场景中深度图像的尺度不变特征变换(SIFT)特征,用SIFT算法与三维模型重建时所用到的一系列2.5维深度图像进行一一匹配,找到与场景中目标姿态最为相似的深度图像,提取此深度图像的三维几何形状特征与模型进行匹配,实现模型的初始化,即将模型重置到与场景目标相接近的姿态。最后用融合二维纹理信息的迭代就近点（ICP）算法实现场景中目标与模型之间的匹配,从而得到场景中三维目标的准确姿态。实验结果验证了方法的可行性与精确性。     

基于多视频的虚实融合系统

提出了一种基于多视频的虚实融合可视化系统的构建方法,旨在将真实世界中的图像和视频融合到虚拟场景中,用视频图像中的纹理和动态信息去丰富虚拟场景,提高虚拟环境的真实性,得到一种增强的虚拟环境.利用无人机采集图像来重建虚拟场景,并借助图像特征点的匹配来实现视频图像的注册.然后利用投影纹理映射技术,将图像投影到虚拟场景中.视频中的动态物体由于在虚拟环境中缺失对应的三维模型,直接投影,当视点发生变化时会产生畸变.首先检测和追踪这些物体,然后尝试使用多种显示方式来解决畸变问题.此外,系统还考虑有重叠区域的多视频之间的融合.实验结果表明,所构造的虚实融合环境是十分有益的.     

基于显著图的输电线路杆塔图像拼接方法

无人机拍摄的输电线路杆塔图像分辨率高且背景复杂,基于传统特征点的拼接算法在背景中检测出大量的特征点增加了图像匹配的时间,影响了杆塔的匹配精度。针对该问题提出了一种既稳定又具有较小时间开销的输电线路杆塔图像自动拼接方法,利用改进的显著性检测算法得到杆塔图像的显著图,将图像的前景与背景分离,减少了背景对图像中杆塔拼接效果的影响;并采用基于定向的加速分割检测特征(FAST)和旋转不变性的二进制鲁棒独立元素特征(BRIEF)描述子(ORB)特征点的图像匹配算法,以提高特征点提取和匹配的速率;最后利用多尺度融合策略得到最终的拼接结果。实验结果表明,所提方法具有较好的拼接效果和拼接效率。     

结合语义分割与模型匹配的室内场景重建方法

目的 由于室内点云场景中物体的密集性、复杂性以及多遮挡等带来的数据不完整和多噪声问题,极大地限制了室内点云场景的重建工作,无法保证场景重建的准确度。为了更好地从无序点云中恢复出完整的场景,提出了一种基于语义分割的室内场景重建方法。方法 通过体素滤波对原始数据进行下采样,计算场景三维尺度不变特征变换（3D scale-invariant feature transform,3D SIFT）特征点,融合下采样结果与场景特征点从而获得优化的场景下采样结果;利用随机抽样一致算法（random sample consensus,RANSAC）对融合采样后的场景提取平面特征,将该特征输入PointNet网络中进行训练,确保共面的点具有相同的局部特征,从而得到每个点在数据集中各个类别的置信度,在此基础上,提出了一种基于投影的区域生长优化方法,聚合语义分割结果中同一物体的点,获得更精细的分割结果;将场景物体的分割结果划分为内环境元素或外环境元素,分别采用模型匹配的方法、平面拟合的方法从而实现场景的重建。结果 在S3DIS （Stanford large-scale 3D indoor space dataset）数据集上进行实验,本文融合采样算法对后续方法的效率和效果有着不同程度的提高,采样后平面提取算法的运行时间仅为采样前的15%;而语义分割方法在全局准确率（overall accuracy,OA）和平均交并比（mean intersection over union,mIoU）两个方面比PointNet网络分别提高了2.3%和4.2%。结论 本文方法能够在保留关键点的同时提高计算效率,在分割准确率方面也有着明显提升,同时可以得到高质量的重建结果。     

基于图像序列的交互式快速建模系统

给出了一个基于图像序列的交互式三维建模系统.通过输入一段未标定的图像或视频序列,系统能够自动地恢复出摄像机参数;然后用户只需要在少量几帧图像上简单勾画出物体的形态结构,系统就能自动解析出多帧之间用户交互的对应关系,从而迅速、逼真地重建出场景的三维模型.该系统提供了点与线段的重建、直线与平面的重建、曲线与曲面的重建等功能,能够满足对现实世界中的复杂场景的快速高精度的重建要求.几组真实拍摄的图像序列的建模实验表明:该系统高效、实用.能够很好地满足实际建模需求.     

点线光流预测机制的图像序列运动直线跟踪

目的 针对复杂场景图像序列中运动直线特征的提取、跟踪问题,提出一种基于点、线光流预测机制的图像序列运动直线跟踪方法。方法 首先根据图像直线的表达式定义点、线光流基本约束方程,由基本约束方程推导出关于点光流与直线光流对应关系的3个重要推论。然后依据点、线光流对应关系,利用图像序列中直线特征上的像素点光流计算直线光流的估计值并根据直线光流阈值筛选图像序列运动直线。最后由筛选出的运动直线及直线光流估计值计算直线的预测坐标并在Hough域内进行跟踪匹配,得到图像序列运动直线跟踪结果。结果 通过合成及真实图像序列实验验证,本文方法能够准确地筛选出图像序列中感兴趣的运动直线,并对运动直线进行稳定地跟踪、匹配,直线跟踪结果未产生干扰直线的误匹配,直线跟踪时间消耗不超过12 s。结论 相对于传统的直线跟踪、匹配方法,本文方法具有较高地直线跟踪精度和较好的鲁棒性,更适用于复杂场景下的运动直线跟踪、匹配问题。     

一种改进的SIFT特征匹配算法

针对尺度不变特征变换(SIFT)特征匹配算法存在计算量大、复杂度高的问题,提出一种基于图像Radon变换的改进SIFT特征匹配算法。改进算法在图像的SIFT特征点采样区域内作d条不同方向的直线,以d条直线上的图像Radon变换作为SIFT特征向量描述符,降低SIFT特征向量的维数,从而提高特征匹配效率。实验结果表明,改进算法具有较高的匹配精度和较少的匹配时间,适用于虚拟场景漫游或目标识别等实时性要求较高的系统。     

Harris相关与特征匹配

基于图像梯度提出一种图像特征,称为Harris相关.应用Harris相关特征,构造图像特征描述子,包括特征点描述子、直线描述子和曲线描述子.这些描述子的构造较简单,对图像旋转与图像亮度的线性变化具有不变性.直线与曲线描述子的构造方法为直线和曲线的匹配提供一种思路.实验表明,应用Harris相关构造的特征点描述子对图像变化表现出较好性能,直线与曲线描述子在实际图像中取得较好的匹配结果.     

虚拟现实场景图像点云大数据展示仿真

虚拟现实技术可以利用计算机图像建模技术构建虚拟空间，通过视觉效果增强现实效果。但是，由于虚拟场景特征点较多，且密度较大，复杂场景的三维重建和展示的难度较高。提出基于图像点云大数据的虚拟现实场景展示方法。利用Kinevt v2设备获取虚拟现实场景两帧图像及点云数据，通过对点云数据的滤波处理，优化虚拟现实场景重构精度。基于滤波处理，提取虚拟现实场景两帧图像的特征点，并匹配特征点，构建三维场景重建模型。将三维场景重构模型与配准后的点云数据、场景颜色信息相融合，完成模型的可视化，实现虚拟现实场景的展示。实验结果证明了研究方法展示的虚拟现实场景特征信息更完整，在图像滤波处理前后，研究方法的虚拟现实场景特征点提取效率均更高，总耗时在7ms以内，且研究方法应用下图像处理器占用的空间内存低于20MB。     

基于二维激光雷达的自动室内三维重建系统

设计了一个基于二维激光雷达的自动室内三维重建系统.系统的硬件由一套自行设计的基于2D激光雷达的三维扫描系统和一台电脑构成.介绍了系统的软件模块,提出了结合最近点迭代(ICP)和通用多边形裁剪(GPC)的3D平面场景合成方法.ICP能够获得不同采集位置之间的位置变化,以此能将各个不同位置获得的3D场景转换到同一坐标系下.场景合成时的碎平面问题通过GPC方法来解决.实验结果表明:该系统成本低,精度高,能稳定可靠地实现室内场景的自动三维重建.     

Automatic reconstruction of 3D human motion pose from uncalibrated monocular video sequences based on markerless human motion tracking

We present a method to reconstruct human motion pose from uncalibrated monocular video sequences based on the morphing appearance model matching. The human pose estimation is made by integrated human joint tracking with pose reconstruction in depth-first order. Firstly, the Euler angles of joint are estimated by inverse kinematics based on human skeleton constrain. Then, the coordinates of pixels in the body segments in the scene are determined by forward kinematics, by projecting these pixels in the scene onto the image plane under the assumption of perspective projection to obtain the region of morphing appearance model in the image. Finally, the human motion pose can be reconstructed by histogram matching. The experimental results show that this method can obtain favorable reconstruction results on a number of complex human motion sequences.     

Real-time indoor scene reconstruction with Manhattan assumption

This paper presents a novel end-to-end system for real-time indoor scene reconstruction, which outperforms traditional image feature point-based method and dense geometry correspondence-based method in handling indoor scenes with less texture and geometry features. In our method, we fully explore the Manhattan assumption, i.e. scenes are majorly consisted with planar surfaces with orthogonal normal directions. Given an input depth frame, we first extract dominant axes coordinates via principle component analysis which involves the orthogonal prior and reduce the influence of noise. Then we calculate the coordinates of dominant planes (such as walls, floor and ceiling) in the coordinates using mean shift. Finally, we compute the camera orientation and reconstruct the scene by proposing a fast scheme based on matching the dominant axes and planes to the previous frame. We have tested our approach on several datasets and demonstrated that it outperforms some well known existing methods in these experiments. The performance of our method is also able to meet the requirement of real-time with an unoptimized CPU implementation.

     

基于RGB-D深度相机的室内场景重建

目的 重建包含真实纹理的彩色场景3维模型是计算机视觉领域重要的研究课题之一,由于室内场景复杂、采样图像序列长且运动无规则,现有的3维重建算法存在重建尺度受限、局部细节重建效果差的等问题。方法 以RGBD-SLAM 算法为基础并提出了两方面的改进,一是将深度图中的平面信息加入帧间配准算法,提高了帧间配准算法的鲁棒性与精度;二是在截断符号距离函数(TSDF)体重建过程中,提出了一种指数权重函数,相比普通的权重函数能更好地减少相机深度畸变对重建的影响。结果 本文方法在相机姿态估计中带来了比RGBD-SLAM方法更好的结果,平均绝对路径误差减少1.3 cm,能取得到更好的重建效果。结论 本文方法有效地提高了相机姿态估计精度,可以应用于室内场景重建中。     

Qualitative Scene Interpretation Using Planar Surfaces

Our aim is to provide an autonomous vehicle moving into an indoor environment with a visual system to perform a qualitative 3D structure reconstruction of the surrounding environment by recovering the different planar surfaces present in the observed scene.The method is based on qualitative detection of planar surfaces by using projective invariant constraints without the use of depth estimates. The goal is achieved by analyzing two images acquired by observing the scene from two different points of view. The method can be applied to both stereo images and motion images.Our method recovers planar surfaces by clustering high variance interest points whose cross ratio measurements are preserved in two different perspective projections. Once interest points are extracted from each image, the clustering process requires to grouping corresponding points by preserving the cross ratio measurements.We solve the twofold problem of finding corresponding points and grouping the coplanar ones through a global optimization approach based on matching of high relational graphs and clustering on the corresponding association graph through a relaxation labeling algorithm.Through our experimental tests, we found the method to be very fast to converge to a solution, showing how higher order interactions, instead to giving rise to a more complex problem, help to speed-up the optimization process and to reach at same time good results.     

Towards a real-time 3D shape reconstruction using a structured light system

This paper deals with 3D shape reconstruction using a structured light system (SLS) which projects a matrix of laser rays onto the scene to be analyzed. The intrinsic problem of such a system is the correspondence problem solving, which in this particular case amounts to matching up the imaged spots and the originating laser rays. In this paper, we propose a method for automatically obtaining configurations of the system (COS) (i.e. the relative positions of the camera, laser projector, and measuring scene) that permit to achieve a direct and unambiguous correspondence. After, we propose a splitting cell algorithm, which efficiently performs a real-time correspondence procedure. Experimental results obtained from both simulated and real data demonstrate that our method provides our SLS with possibilities for real-time applications.     

Automatic 3D Indoor Scene Updating with RGBD Cameras

Since indoor scenes are frequently changed in daily life, such as re‐layout of furniture, the 3D reconstructions for them should be flexible and easy to update. We present an automatic 3D scene update algorithm to indoor scenes by capturing scene variation with RGBD cameras. We assume an initial scene has been reconstructed in advance in manual or other semi‐automatic way before the change, and automatically update the reconstruction according to the newly captured RGBD images of the real scene update. It starts with an automatic segmentation process without manual interaction, which benefits from accurate labeling training from the initial 3D scene. After the segmentation, objects captured by RGBD camera are extracted to form a local updated scene. We formulate an optimization problem to compare to the initial scene to locate moved objects. The moved objects are then integrated with static objects in the initial scene to generate a new 3D scene. We demonstrate the efficiency and robustness of our approach by updating the 3D scene of several real‐world scenes.     

基于图像运动的三维重建及虚拟化应用

提出了一种稳定、快速地获取摄像机视频运动图像的三维重建方法,并对该运动图像做适当的虚拟化处理以展示重建效果。采用基于尺度不变特征点匹配的摄像机标定进行三维重建。尺度不变特征对于视频图像中的特征具有优秀敏锐的匹配能力,极大地放宽了摄像机标定对于设备上的限制,拓宽了实时三维重建的适用范围。通过对系统的一系列优化,不但提升了三维重建的精度,减少了错误匹配对摄像机标定的影响,而且进一步提升了处理速度。通过在三维重建的基础之上进行虚拟化处理,展示了本系统的三维重建效果。实验结果表明,该系统适用范围广,处理速度较快,重建精度高,实现了基于视频运动图像的三维重建。     

Radio propagation modeling and real test of ZigBee based indoor wireless sensor networks

The deployment of nodes in Wireless Sensor Networks (WSNs) arises as one of the biggest challenges of this field, which involves in distributing a large number of embedded systems to fulfill a specific application. The connectivity of WSNs is one of the main issues to assure the efficiency of the system implementation and the quality of the service of the deployment, which is difficult to estimate due to the diversity and irregularity of the applied environment and it affects the WSN designers’ decision on deploying sensor nodes. Therefore, in this paper, a new method is proposed to enhance the efficiency and accuracy on ZigBee propagation modeling and simulation in indoor environments. The method consists of two steps: automatic 3D indoor reconstruction and 3D ray-tracing based radio simulation. The automatic 3D indoor reconstruction employs unattended image classification algorithm and image vectorization algorithm to accurately build the environment database, which also significantly reduces time and efforts spent on non-radio propagation issues. The 3D ray tracing is developed by using a kd-tree space division algorithm and a modified polar sweep algorithm, which accelerates the searching of rays over the entire space. A ZigBee signal propagation model is proposed for the ray-tracing engine by considering both the materials of obstacles and the impact of positions along the ray path of the radio. Three different WSN deployments are realized in the indoor environment of an office and the simulation results are verified to be accurate. Experimental results also indicate that the proposed method is efficient in the pre-simulation strategy and the 3D ray searching scheme, and it is robust for different indoor environments.