A comparison of volumetric information gain metrics for active 3D object reconstruction

In this paper, we investigate the following question: when performing next best view selection for volumetric 3D reconstruction of an object by a mobile robot equipped with a dense (camera-based) depth sensor, what formulation of information gain is best? To address this question, we propose several new ways to quantify the volumetric information (VI) contained in the voxels of a probabilistic volumetric map, and compare them to the state of the art with extensive simulated experiments. Our proposed formulations incorporate factors such as visibility likelihood and the likelihood of seeing new parts of the object. The results of our experiments allow us to draw some clear conclusions about the VI formulations that are most effective in different mobile-robot reconstruction scenarios. To the best of our knowledge, this is the first comparative survey of VI formulation performance for active 3D object reconstruction. Additionally, our modular software framework is adaptable to other robotic platforms and general reconstruction problems, and we release it open source for autonomous reconstruction tasks.     

Symmetry-seeking models and 3D object reconstruction

We propose models of 3D shape which may be viewed as deformable bodies composed of simulated elastic material. In contrast to traditional, purely geometric models of shape, deformable models are active—their shapes change in response to externally applied forces. We develop a deformable model for 3D shape which has a preference for axial symmetry. Symmetry is represented even though the model does not belong to a parametric shape family such as (generalized) cylinders. Rather, a symmetry-seeking property is designed into internal forces that constrain the deformations of the model. We develop a framework for 3D object reconstruction based on symmetry-seeking models. Instances of these models are formed from monocular image data through the action of external forces derived from the data. The forces proposed in this paper deform the model in space so that the shape of its projection into the image plane is consistent with the 2D silhouette of an object of interest. The effectiveness of our approach is demonstrated using natural images.     

立体视频对象分割及其三维重建算法研究*

为更加有效分析立体视频对象,本文提出了一种基于离散冗余小波变换的立体视频对象分割算法,首先采用离散冗余小波变换提取特征点结合DT网格技术的视差估计方法,获得了可靠的视差场,再利用视差信息对立体视频中静止对象进行分割。对于立体视频序列中的运动对象,采用离散冗余小波提取运动区域的方法进行分割。实验结果表明,本算法对有重叠的多视频对象具有较好的分割效果,可同时分割静止物体和运动物体,具有较好的精确性和鲁棒性。对于分割出的立体视频对象,结合深度信息对其进行三维重建,得到较好的三维效果。     

用单数码相机实现物体表面的三维重建

根据由运动重建物体结构的原理,设计了一个简便易操作的三维重建系统,具体做法是:先用张氏标定法求得内参数矩阵,然后在两个不同的未知位置拍摄物体得到两幅图像,经立体匹配后,利用图像特征点的对应关系求解基本矩阵和本质矩阵,分解本质矩阵获得两个拍摄位置确定的摄像机运动参数(旋转矩阵和平移向量),进而求出相机在两个位置的投影矩阵,最后用三角法计算出物体表面特征点的三维坐标并在OpenGL中重建物体表面.和传统的立体视觉系统相比,本系统只需要一台数码相机和平面方格模板就可以实现三维重建,因此适用于普通相机用户.     

Automatic 3D object segmentation in multiple views using volumetric graph-cuts

We propose an algorithm for automatically obtaining a segmentation of a rigid object in a sequence of images that are calibrated for camera pose and intrinsic parameters. Until recently, the best segmentation results have been obtained by interactive methods that require manual labelling of image regions. Our method requires no user input but instead relies on the camera fixating on the object of interest during the sequence. We begin by learning a model of the object’s colour, from the image pixels around the fixation points. We then extract image edges and combine these with the object colour information in a volumetric binary MRF model. The globally optimal segmentation of 3D space is obtained by a graph-cut optimisation. From this segmentation an improved colour model is extracted and the whole process is iterated until convergence.     

由单幅二维灰度图像重构物体表面形状*

鉴于目前很少有论文讨论完整的由单幅二维灰度图像重构物体表面形状的算法,包括它的控制参数的估计及算法的实现,介绍了一种完整的SFS算法.它在考虑自遮掩影响的情况下,有效地估计了SFS算法中涉及的各种控制参数,并引入亮度约束、灰度梯度约束和可积性约束,计算出表面高度和表面向量,实现三维重构.与传统的算法相比,本算法无论是在速度还是在精度方面都达到了比较高的水平,具有一定的应用前景.最后还指出了在MATLAB中实现需要注意的问题.     

Automated 3D volumetric reconstruction of multiple-room building interiors for as-built BIM

Currently, fully automated as-built modeling of building interiors using point-cloud data still remains an open challenge, due to several problems that repeatedly arise: (1) complex indoor environments containing multiple rooms; (2) time-consuming and labor-intensive noise filtering; (3) difficulties of representation of volumetric and detail-rich objects such as windows and doors. This study aimed to overcome such limitations while improving the amount of details reproduced within the model for further utilization in BIM. First, we input just the registered three-dimensional (3D) point-cloud data and segmented the point cloud into separate rooms for more effective performance of the later modeling phases for each room. For noise filtering, an offset space from the ceiling height was used to determine whether the scan points belonged to clutter or architectural components. The filtered points were projected onto a binary map in order to trace the floor-wall boundary, which was further refined through subsequent segmentation and regularization procedures. Then, the wall volumes were estimated in two ways: inside- and outside-wall-component modeling. Finally, the wall points were segmented and projected onto an inverse binary map, thereby enabling detection and modeling of the hollow areas as windows or doors. The experimental results on two real-world data sets demonstrated, through comparison with manually-generated models, the effectiveness of our approach: the calculated RMSEs of the two resulting models were 0.089 m and 0.074 m, respectively.     

Plane-based optimization for 3D object reconstruction from single line drawings

In previous optimization-based methods of 3D planar-faced object reconstruction from single 2D line drawings, the missing depths of the vertices of a line drawing (and other parameters in some methods) are used as the variables of the objective functions. A 3D object with planar faces is derived by finding values for these variables that minimize the objective functions. These methods work well for simple objects with a small number N of variables. As N grows, however, it is very difficult for them to find expected objects. This is because with the nonlinear objective functions in a space of large dimension N, the search for optimal solutions can easily get trapped into local minima. In this paper, we use the parameters of the planes that pass through the planar faces of an object as the variables of the objective function. This leads to a set of linear constraints on the planes of the object, resulting in a much lower dimensional nullspace where optimization is easier to achieve. We prove that the dimension of this nullspace is exactly equal to the minimum number of vertex depths which define the 3D object. Since a practical line drawing is usually not an exact projection of a 3D object, we expand the nullspace to a larger space based on the singular value decomposition of the projection matrix of the line drawing. In this space, robust 3D reconstruction can be achieved. Compared with two most related methods, our method not only can reconstruct more complex 3D objects from 2D line drawings, but also is computationally more efficient.     

3D object reconstruction by point coordinates of 2D line drawings

It is a research subject in computer vision to 3D reconstruction of an object represented by a single 2D line drawing.Previous works on 3D reconstruction from 2D line drawings focus on objects with lines,plane,view,and so on.This paper mainly studies the 3D reconstruction from 2D line drawings.Besides,a new approach is proposed:it is that for the research of the point coordinates of 2D line drawings,so as to achieve the object reconstruction by the reconstruction of point coordinates.The reconstruction process includes:① the collection of point coordinates (X,Y) of 2D line drawings;② the derivation of mathematical formula about the reconstruction of the point of 2D line drawings,and calculating the corresponding point of the 3D coordinates;③ the regeneration of 3D graphics with 3D points;④ analyze error by the proportional of parallel of axonometric projection,in order to prove the accuracy of the method.     

单幅图像三维表面重建的算法研究与实现

以单幅二维图像为研究对象,应用SFS的基本理论,首先对二维图像进行了光照的倾角和偏角的估计,在光源坐标系下计算出物体每一点的表面梯度;之后,旋转坐标系,在观察坐标系下得到物体的表面梯度;最后采用三点辛普森公式求出物体表面的高度。整个算法流程简单,求解层次分明,重构满意度较高。     

光截法旋转式物体三维重构

本文依据计算机视觉的理论,采用光截法,搭建了一个旋转物体三维重构实验平台。探讨了摄像机及回转台中心轴的标定和空间坐标转换还原工作。并以表面不平的小纸杯为对象进行了具体实验,在MATLAB下对其进行了图像处理获得物体空间三维数据,并将其三维显示出来,获得了较满意的结果。     

三维物体表面重建的分支处理算法研究

根据平面点集Delaunay三角剖分的特性,将Delaunay三角剖分应用到分支问题上,改进和实现了一种分支问题处理算法。将相邻层轮廓线投影到同一个剖面上形成一个带约束边的平面点集,并将它们Delaunay三角化,根据这些三角形组来生成新的轮廓线,使轮廓线一一对应。实验结果表明该算法实现的效果较符合实际情况,能有效地处理各种不同情况。     

可扩展的多分辨率三维地形应用框架的研究

多分辨率建模作为解决大面积地形实时动态显示的手段，近年来在国内外得到普遍的重视和研究，针对不规则三角网格和多分辨率模型的特点，总结前人在这方面的有关算法和数据结构，对构建可扩展的多分辨率地形应用框架做了初步研究。     

Decomposition of complex line drawings with hidden lines for 3D planar-faced manifold object reconstruction

Three-dimensional object reconstruction from a single 2D line drawing is an important problem in computer vision. Many methods have been presented to solve this problem, but they usually fail when the geometric structure of a 3D object becomes complex. In this paper, a novel approach based on a divide-and-conquer strategy is proposed to handle the 3D reconstruction of a planar-faced complex manifold object from its 2D line drawing with hidden lines visible. The approach consists of four steps: 1) identifying the internal faces of the line drawing, 2) decomposing the line drawing into multiple simpler ones based on the internal faces, 3) reconstructing the 3D shapes from these simpler line drawings, and 4) merging the 3D shapes into one complete object represented by the original line drawing. A number of examples are provided to show that our approach can handle 3D reconstruction of more complex objects than previous methods.     

Multiresolution video object extraction fitted to scalable wavelet-based object coding

To enable content based functionalities in video processing algorithms, decomposition of scenes into semantic objects is necessary. A semi-automatic Markov random field based multiresolution algorithm is presented for video object extraction in a complex scene. In the first frame, spatial segmentation and user intervention determine objects of interest. The specified objects are subsequently tracked in successive frames and newly appeared objects/regions are also detected. The video object extraction algorithm includes discrete wavelet transform decomposition multiresolution Markov random field (MRF)-based spatial segmentation with emphasis on border smoothness at different resolutions, and an MRF-based backward region classification that determines the tracked objects in the scene. Finally, a motion constraint, embedded in the region classifier, determines the newly appeared objects/regions and completes the proposed algorithm towards an efficient video segmentation algorithm. The results are applicable for generic segmentation applications, however the proposed multiresolution video segmentation algorithm supports scalable object-based wavelet coding in particular. Moreover, compared to traditional object extraction algorithms, it produces smoother and more visually pleasing shape masks at different resolutions. The proposed effective multiresolution video object extraction method allows for larger motion, better noise tolerance and less computational complexity     

Stereo VoVNet-CNN for 3D object detection

Multimedia Tools and Applications - 3D object detection is a key issue and research in autonomous vehicle and computer vision. 3D detection methods based on stereoscopic images estimate 3D boxes...     

Single image 3D object reconstruction based on deep learning: A review

Multimedia Tools and Applications - The reconstruction of 3D object from a single image is an important task in the field of computer vision. In recent years, 3D reconstruction of single image...     

Target recognition via 3D object reconstruction from image sequence and contour matching

This paper proposes an automatic target recognition (ATR) system based on the three-dimensional (3D) reconstruction of the target from an image sequence. The main contribution of this work is twofold: (1) we present a modified voxel coloring reconstruction algorithm and (2) we employ the 3D reconstructed target model to generate the front and side target templates at zero depression angle to be used in the target recognition process. Target recognition is performed by matching the generated templates to a library using a subpixel contour matching algorithm. Experimental results on simulated scenes show the accuracy of the approach presented in this paper.     

Information entropy-based viewpoint planning for 3-D object reconstruction

In this paper, we present an information entropy-based viewpoint-planning approach for reconstruction of freeform surfaces of three-dimensional objects. To achieve the reconstruction, the object is first sliced into a series of cross section curves, with each curve to be reconstructed by a closed B-spline curve. In the framework of Bayesian statistics, we propose an improved Bayesian information criterion (BIC) for determining the B-spline model complexity. Then, we analyze the uncertainty of the model using entropy as the measurement. Based on this analysis, we predict the information gain for each cross section curve for the next measurement. After predicting the information gain of each curve, we obtain the information change for all the B-spline models. This information gain is then mapped into the view space. The viewpoint that contains maximal information gain about the object is selected as the next best view. Experimental results show successful implementation of our view planning method for digitization and reconstruction of freeform objects.