Multi-stage 3D reconstruction under circular motion

In this paper, we propose a new method for 3D reconstruction from an image sequence captured by a camera with constant intrinsic parameters undergoing circular motion. We introduce a method, called circular projective reconstruction, for enforcing the circular constraint in a factorization-based projective reconstruction. To deal with the missing data problem, our method uses a multi-stage approach to reconstructing the objects and cameras, which first computes a circular projective reconstruction of a sub-sequence and then extends the reconstruction to the complete sequence. Camera matrix, rotation angles, and 3D structure are computed iteratively in a way that the 2D reprojection error is minimized. The algorithm is evaluated using real image sequences.     

一种非定标图像高精度三维重建算法

由非定标图像重建三维场景有着广泛的应用。给出了一种非定标多视图像三维重建算法。该算法主要基于因子分解和光束法平差技术。首先用因子分解方法得到射影空间下相机投影矩阵和物点坐标,以旋转矩阵的正交性以及对偶绝对二次曲面秩为3为约束,将射影空间升级到欧式空间,最后用光束法平差进行优化。该方法可同时获得相机的内外参数、畸变系数和场景的三维坐标。仿真实验表明,在1000 mm×1000 mm×400mm的范围内,当像点检测误差在0-1pixel和0-2pixel内,所重建三维点的误差分别为0.1530 mm和0.6712 mm。在500 mm×500 m×200 mm下,真实实验重构三维点的误差在0.3 mm以内。所提出的算法稳定可靠,可对实际工程进行指导。     

基于未标定图像的三维重建算法

提出一种基于多幅未标定图像的三维重建算法。在标记点匹配的基础上进行射影重建,通过施加度量约束将射影重建升级为欧氏重建,即利用未标定的透视图像恢复相机的内、外部参数以及标记点的三维空间坐标,实现场景的三维重建。标记点易于进行点对精确匹配,较手动拾取匹配提高了效率。实验结果表明,利用该算法能够大幅减小再投影误差。     

Camera calibration using identical objects

This paper describes a method for camera calibration using identical products. In this paper, we postulate an imaginative rigid motion between any two identical products, and the imaginative rigid motion could offer a pair of circular points. As is known, three pairs of projections of the circular points are needed to result in the closed-form solution for calibration. In our method, we obtain three pairs of projections of the circular points from only two images of three identical products, or three images of two identical products. When only two identical products are utilized, our method is almost the dual of the stereo calibration from rigid motions. A direct approach is taken here instead of the two-step process in stereo calibration. Furthermore, a better projective reconstruction could be performed from the estimation of the camera parameters to avoid the dominant projective-to-affine error in the stereo calibration. Finally, we conduct a nonlinear refinement based on the maximum likelihood estimation. The experimental results from synthetic data and real data prove our method convenient and robust to noise.     

Multiview constraints on homographies

The image motion of a planar surface between two camera views is captured by a homography (a 2D projective transformation). The homography depends on the intrinsic and extrinsic camera parameters, as well as on the 3D plane parameters. While camera parameters vary across different views, the plane geometry remains the same. Based on this fact, we derive linear subspace constraints on the relative homographies of multiple (⩾ 2) planes across multiple views. The paper has three main contributions: 1) We show that the collection of all relative homographies (homologies) of a pair of planes across multiple views, spans a 4-dimensional linear subspace. 2) We show how this constraint can be extended to the case of multiple planes across multiple views. 3) We show that, for some restricted cases of camera motion, linear subspace constraints apply also to the set of homographies of a single plane across multiple views. All the results derived are true for uncalibrated cameras. The possible utility of these multiview constraints for improving homography estimation and for detecting nonrigid motions are also discussed     

摄像机内参数自标定——理论与算法

讨论如何通过摄像机的旋转运动标定其内参数.当摄像机绕其坐标轴旋转时,运用 代数方法给出了计算内参数的公式.该公式在2D投影变换接近理论值P时是非常实用的. 在摄像机绕未知轴旋转时,根据相应的2D投影变换,运用矩阵特征向量理论给出了内参数的 通解公式.通过摄像机绕两个不同未知轴的旋转,摄像机内参数能被唯一地确定.这些结果为 摄像机自标定算法提供了理论基础,同时也给出了实用性算法.模拟实验和真实图像实验的 结果表明本文所给的算法具有一定实用价值.     

Self-calibration of a 1D projective camera and its application to the self-calibration of a 2D projective camera

We introduce the concept of self-calibration of a 1D projective camera from point correspondences, and describe a method for uniquely determining the two internal parameters of a 1D camera, based on the trifocal tensor of three 1D images. The method requires the estimation of the trifocal tensor which can be achieved linearly with no approximation unlike the trifocal tensor of 2D images and solving for the roots of a cubic polynomial in one variable. Interestingly enough, we prove that a 2D camera undergoing planar motion reduces to a 1D camera. From this observation, we deduce a new method for self-calibrating a 2D camera using planar motions. Both the self-calibration method for a 1D camera and its applications for 2D camera calibration are demonstrated on real image sequences.     

摄像机内参数自标定——理论与算法1)

讨论如何通过摄像机的旋转运动标定其内参数.当摄像机绕其坐标轴旋转时,运用代数方法给出了计算内参数的公式.该公式在2D投影变换接近理论值P时是非常实用的.在摄像机绕未知轴旋转时,根据相应的2D投影变换,运用矩阵特征向量理论给出了内参数的通解公式.通过摄像机绕两个不同未知轴的旋转,摄像机内参数能被唯一地确定.这些结果为摄像机自标定算法提供了理论基础,同时也给出了实用性算法。模拟实验和真实图像实验的结果表明本文所给的算法具有一定实用价值.     

Multiple motion scene reconstruction with uncalibrated cameras

In this paper, we describe a reconstruction method for multiple motion scenes, which are scenes containing multiple moving objects, from uncalibrated views. Assuming that the objects are moving with constant velocities, the method recovers the scene structure, the trajectories of the moving objects, the camera motion, and the camera intrinsic parameters (except skews) simultaneously. We focus on the case where the cameras have unknown and varying focal lengths while the other intrinsic parameters are known. The number of the moving objects is automatically detected without prior motion segmentation. The method is based on a unified geometrical representation of the static scene and the moving objects. It first performs a projective reconstruction using a bilinear factorization algorithm and, then, converts the projective solution to a Euclidean one by enforcing metric constraints. Experimental results on synthetic and real images are presented.     

计算机视觉测量系统的误差模型分析

基于相机的钻孔模型，研究了相机光学系统产生的4种非线性误差，并在有径向失真和图像中心偏移失真的情况下，用模拟数据对匹配误差和不同量化等级在重建过程中的传播过程和影响作了详细讨论，为视觉测量系统达到高精度的三维重建提供了可靠的误差模型。     

Simplified Active Calibration

We present a new mathematical formulation to estimate the intrinsic parameters of a camera in active or robotic platforms. We show that the focal lengths can be estimated using only one point correspondence that relates images taken before and after a degenerate rotation of the camera. The estimated focal lengths are then treated as known parameters to obtain a linear set of equations to calculate the principal point. Assuming that the principal point is close to the image center, the accuracy of the linear equations are increased by integrating the image center into the formulation. We extensively evaluate the formulations on a simulated camera, 3D scenes and real-world images. Our error analysis over simulated and real images indicates that the proposed Simplified Active Calibration method estimates the parameters of a camera with low error rates that can be used as an initial guess for further non-linear refinement procedures. Simplified Active Calibration can be employed in real-time environments for automatic calibrations given the proposed closed-form solutions.     

Autocalibration with the Minimum Number of Cameras with Known Pixel Shape

In 3D reconstruction, the recovery of the calibration parameters of the cameras is paramount since it provides metric information about the observed scene, e.g., measures of angles and ratios of distances. Autocalibration enables the estimation of the camera parameters without using a calibration device, but by enforcing simple constraints on the camera parameters. In the absence of information about the internal camera parameters such as the focal length and the principal point, the knowledge of the camera pixel shape is usually the only available constraint. Given a projective reconstruction of a rigid scene, we address the problem of the autocalibration of a minimal set of cameras with known pixel shape and otherwise arbitrarily varying intrinsic and extrinsic parameters. We propose an algorithm that only requires 5 cameras (the theoretical minimum), thus halving the number of cameras required by previous algorithms based on the same constraint. To this purpose, we introduce as our basic geometric tool the six-line conic variety (SLCV), consisting in the set of planes intersecting six given lines of 3D space in points of a conic. We show that the set of solutions of the Euclidean upgrading problem for three cameras with known pixel shape can be parameterized in a computationally efficient way. This parameterization is then used to solve autocalibration from five or more cameras, reducing the three-dimensional search space to a two-dimensional one. We provide experiments with real images showing the good performance of the technique.     

Non-Rigid Stereo Factorization

In this paper we address the problem of recovering 3D non-rigid structure from a sequence of images taken with a stereo pair. We have extended existing non-rigid factorization algorithms to the stereo camera case and presented an algorithm to decompose the measurement matrix into the motion of the left and right cameras and the 3D shape, represented as a linear combination of basis-shapes. The added constraints in the stereo camera case are that both cameras are viewing the same structure and that the relative orientation between both cameras is fixed. Our focus in this paper is on the recovery of flexible 3D shape rather than on the correspondence problem. We propose a method to compute reliable 3D models of deformable structure from stereo images. Our experiments with real data show that improved reconstructions can be achieved using this method. The algorithm includes a non-linear optimization step that minimizes image reprojection error and imposes the correct structure to the motion matrix by choosing an appropriate parameterization. We show that 3D shape and motion estimates can be successfully disambiguated after bundle adjustment and demonstrate this on synthetic and real image sequences. While this optimization step is proposed for the stereo camera case, it can be readily applied to the case of non-rigid structure recovery using a monocular video sequence. Electronic supplementary material Electronic supplementary material is available for this article at and accessible for authorised users.     

Structure-from-motion using lines: Representation, triangulation, and bundle adjustment

We address the problem of camera motion and 3D structure reconstruction from line correspondences across multiple views, from initialization to final bundle adjustment. One of the main difficulties when dealing with line features is their algebraic representation. First, we consider the triangulation problem. Based on Plücker coordinates to represent the 3D lines, we propose a maximum likelihood algorithm, relying on linearizing the Plücker constraint and on a Plücker correction procedure, computing the closest Plücker coordinates to a given 6-vector. Second, we consider the bundle adjustment problem, which is essentially a nonlinear optimization process on camera motion and 3D line parameters. Previous overparameterizations of 3D lines induce gauge freedoms and/or internal consistency constraints. We propose the orthonormal representation, which allows handy nonlinear optimization of 3D lines using the minimum four parameters with an unconstrained optimization engine. We compare our algorithms to existing ones on simulated and real data. Results show that our triangulation algorithm outperforms standard linear and bias-corrected quasi-linear algorithms, and that bundle adjustment using our orthonormal representation yields results similar to the standard maximum likelihood trifocal tensor algorithm, while being usable for any number of views.     

一种优化的消失点估计方法及误差分析

空间一组平行直线在图像平面上所成的像的交点称为消失点. 消失点可以提供大量的场景三维结构信息. 本文提出一种新的优化的消失点估计方法. 该方法基于随机采样一致算法(Random sample consensus, RANSAC)对图像空间中的线段进行聚类, 通过最小化Sampson误差获得消失点的极大似然估计(Maximum likelihood estimation, MLE). 该方法不需要预知摄像机参数及直线的三维位置信息. 为了对该算法进行定量评估, 构造了基于反向传播的消失点误差传递模型. 实验结果验证了本文提出算法的有效性.     

采用棋盘格靶标的全自动相机标定方法

为了去除相机标定过程中的人为干预,提出了一种采用改进的棋盘格靶标的全自动相机标定方法。识别出每幅标定图像中的四个标志圆,利用四个标志圆圆心的图像坐标和物理坐标计算二维射影变换矩阵。依据该射影变换矩阵计算出棋盘格角点的初始图像位置,接着提取亚像素精度的角点位置。迭代求解需要标定的相机参数。由实验可知,该全自动相机标定方法的棋盘格角点识别能力和相机标定精度,与Bouguet的相机标定工具箱相当,且可以显著地减少标定时间和工作量。利用20幅分辨率为640×480的靶标图像标定相机仅需16 s。     

Auto‐calibration of a projector–camera stereo system for projection mapping

Standard camera and projector calibration techniques use a checkerboard that is manually shown at different poses to determine the calibration parameters. Furthermore, when image geometric correction must be performed on a three‐dimensional (3D) surface, such as projection mapping, the surface geometry must be determined. Camera calibration and 3D surface estimation can be costly, error prone, and time‐consuming when performed manually. To address this issue, we use an auto‐calibration technique that projects a series of Gray code structured light patterns. These patterns are captured by the camera to build a dense pixel correspondence between the projector and camera, which are used to calibrate the stereo system using an objective function, which embeds the calibration parameters together with the undistorted points. Minimization is carried out by a greedy algorithm that minimizes the cost at each iteration with respect to both calibration parameters and noisy image points. We test the auto‐calibration on different scenes and show that the results closely match a manual calibration of the system. We show that this technique can be used to build a 3D model of the scene, which in turn with the dense pixel correspondence can be used for geometric screen correction on any arbitrary surface.     

Visualizing multiple error-sensitivity fields for single camera positioning

In many data acquisition tasks, the placement of a real camera can vary significantly in complexity from one scene to another. Optimal camera positioning should be governed not only by least error sensitivity, but in addition to real-world practicalities given by various physical, financial and other types of constraints. It would be a laborious and costly task to model all these constraints if one were to rely solely on fully automatic algorithms to make the decision. In this work, we present a study using 2D and 3D visualization methods to assist in single camera positioning based on error sensitivity of reconstruction and other physical and financial constraints. We develop a collection of visual mappings that depict the composition of multiple error sensitivity fields that occur for a given camera position. Each camera position is then mapped to a 3D visualization that enables visual assessment of the camera configuration. We find that the combined 2D and 3D visualization effectively aids the estimation of camera placement without the need for extensive manual configuration through trial and error. Importantly, it still provides the user with sufficient flexibility to make dynamic decisions based on physical and financial constraints that can not be encoded easily in an algorithm. We demonstrate the utility of our system on two real-world applications namely snooker analysis and camera surveillance.     

Projective texture atlas construction for 3D photography

The use of attribute maps for 3D surfaces is an important issue in geometric modeling, visualization and simulation. Attribute maps describe various properties of a surface that are necessary in applications. In the case of visual properties, such as color, they are also called texture maps. Usually, the attribute representation exploits a parametrization g:U??²→?³ of a surface in order to establish a two-dimensional domain where attributes are defined. However, it is not possible, in general, to find a global parametrization without introducing distortions into the mapping. For this reason, an atlas structure is often employed. The atlas is a set of charts defined by a piecewise parametrization of a surface, which allows local mappings with small distortion. Texture atlas generation can be naturally posed as an optimization problem where the goal is to minimize both the number of charts and the distortion of each mapping. Additionally, specific applications can impose other restrictions, such as the type of mapping. An example is 3D photography, where the texture comes from images of the object captured by a camera [4]. Consequently, the underlying parametrization is a projective mapping. In this work, we investigate the problem of building and manipulating texture atlases for 3D photography applications. We adopt a variational approach to construct an atlas structure with the desired properties. For this purpose, we have extended the method of Cohen–Steiner et al. [6] to handle the texture mapping set-up by minimizing distortion error when creating local charts. We also introduce a new metric tailored to projective maps that is suited to 3D photography.     

基于平面格网的非量测CCD相机自标定

传统的相机标定方法通常需要建立复杂3维标定块或高精度3维控制场,在实际应用中受到了一定的限制。本文采用平面控制格网作为标定块,根据相机的理想模型确定内方位元素,利用2维直接线性变换和共线方程分解出相机的外方位元素初值,采用改进的Hough变换算法检测标定图像中的格网直线并利用最小二乘法拟合出最佳直线,通过求直线的交点得到标定格网点的像坐标。最后利用自检校光线束法平差进行相机的精确标定。实际图像数据实验结果表明,主点和焦距的标定精度分别达到了0.2像素和0.3像素左右。可以满足高精度近景3维量测的要求。