A Unified Theory of Uncalibrated Stereo for Both Perspective and Affine Cameras

This paper addresses the recovery of structure and motion from uncalibrated images of a scene under full perspective or under affine projection. Particular emphasis is placed on the configuration of two views, while the extension to $N$ views is given in Appendix. A unified expression of the fundamental matrix is derived which is valid for any projection model without lens distortion (including full perspective and affine camera). Affine reconstruction is considered as a special projective reconstruction. The theory is elaborated in a way such that everyone having knowledge of linear algebra can understand the discussion without difficulty. A new technique for affine reconstruction is developed, which consists in first estimating the affine epipolar geometry and then performing a triangulation for each point match with respect to an implicit common affine basis.     

Uncalibrated Motion Capture Exploiting Articulated Structure Constraints

We present an algorithm for 3D reconstruction of dynamic articulated structures, such as humans, from uncalibrated multiple views. The reconstruction exploits constraints associated with a dynamic articulated structure, specifically the conservation over time of length between rotational joints. These constraints admit reconstruction of metric structure from at least two different images in each of two uncalibrated parallel projection cameras. As a by product, the calibration of the cameras can also be computed. The algorithm is based on a stratified approach, starting with affine reconstruction from factorization, followed by rectification to metric structure using the articulated structure constraints. The exploitation of these specific constraints admits reconstruction and self-calibration with fewer feature points and views compared to standard self-calibration. The method is extended to pairs of cameras that are zooming, where calibration of the cameras allows compensation for the changing scale factor in a scaled orthographic camera. Results are presented in the form of stick figures and animated 3D reconstructions using pairs of sequences from broadcast television. The technique shows promise as a means of creating 3D animations of dynamic activities such as sports events.     

Affine shape representation from motion through reference points

This paper shows how an affine representation of spatial configuration is obtained from a pair of projection views. Calibration of cameras and knowledge of the camera's motion are not necessary; however, some preselected reference points and their correspondences are needed. Projective and affine geometry invariants are trickily manipulated to do the affine reconstruction. The method is thus geometrically constructive. When it is compared with the solution proposed in 1989 by J.J. Koenderink and A.J. Van Doorn (Affine Structure from Motion, Technical Report, Utrect University), the method provides a viewpoint-independent affine representation under parallel projections. Further, we investigate the central-projection case in which, with three additional special reference points, the same affine reconstruction can be done. We also discuss some important applications of this viewpoint independence of shape representation.     

Affine structure from line correspondences with uncalibrated affinecameras

This paper presents a linear algorithm for recovering 3D affine shape and motion from line correspondences with uncalibrated affine cameras. The algorithm requires a minimum of seven line correspondences over three views. The key idea is the introduction of a one-dimensional projective camera. This converts 3D affine reconstruction of “line directions” into 2D projective reconstruction of “points”. In addition, a line-based factorization method is also proposed to handle redundant views. Experimental results both on simulated and real image sequences validate the robustness and the accuracy of the algorithm     

Depth recovery and affine reconstruction under camera pure translation

This paper addresses the problems of depth recovery and affine reconstruction from two perspective images, which are generated by an uncalibrated translating camera. Firstly, we develop a new constraint that the homography for the plane, which is orthogonal to the optical axis, is determined only by the epipole and the plane's relative distance to the origin under camera pure translation. The algorithm of depth recovery is based on this new constraint, and it can successfully avoid the step of camera calibration. With the recovered depth, we show that affine reconstruction can be obtained readily. The proposed affine reconstruction does not need any control points, which were used to expand the affine coordinate system in existing method. Therefore, it could avoid the step of non-planarity verification as well as the errors from the control points. Error analysis is also presented to evaluate the uncertainty for the recovered depth value. Finally, we have tested the proposed algorithm with both simulated data and real image data. And the results show that the proposed algorithm is accurate and practical.     

基于仿射点对应的分层重构

提出了一种基于仿射点对应的分层重构方法,所谓仿射点对应是指相差一个仿射变换的两个空间点集的图像对应．该方法主要分为以下三个步骤：首先,从点对应计算准仿射重构;然后,由仿射点对应的准仿射重构建立一个三维射影变换,并利用这个射影变换的特征向量来确定无穷远平面,从而得到仿射重构;最后,从仿射重构所获得的无穷远平面单应矩阵标定摄像机内参数,进而得到度量重构．在上述三个步骤中,第二个步骤是最关键的,即如何确定对应于无穷远平面的特征向量,这也是该文的新思想和主要贡献所在．仿真和真实图像实验均表明,该文的方法是有效的,并且有很好的鲁棒性．     

Compression of Probabilistic Volumetric Models using multi-resolution scene flow

This paper presents a novel method to estimate dense scene flow using volumetric and probabilistic 3-d models. The method first reconstructs 3-d models at each time step using images synchronously captured from multiple views. Then, the 3-d motion between two consecutive 3-d models is estimated using a formulation that is the analog of Horn and Schunck's optical flow method. This particular choice of 3-d model representation allows estimating highly dense scene flow results, tracking of surfaces undergoing topological change and reliably recovering large motion displacements. The benefits of the method and the accuracy of 3-d flow results are demonstrated on recent multi-view datasets. The second goal of this work is to compress and reconstruct 3-d scenes at various time points using the estimated flow. A new method of scene warping is proposed that involves partitioning the optical flow field in regions of coherent motion which are subsequently parametrized by affine transformations. The compression objective of this work is achieved by the low storage requirements of the affine parameters that describe the optical flow field and the efficient reconstruction method through warping.     

基于平面与直线的仿射重建

首先给出了无穷远平面的单应矩阵以及仿射重建算法，然后从数学上严格证明了下述命题：在变参数模型下，如果场景中含有一张平面和一对平行直线，或者场景中含有两张平行平面，则从两个平移视点下的图像均可以线性地对场景进行仿射重建；文章同时指出：如果场景中包含一对平行平面和一对平行直线，则从两个一般运动视点也可以线性地重建场景的仿射几何．大量的模拟和真实图像实验表明，该线性仿射重建算法是正确的，同时具有较高的重建精度和鲁棒性．     

Combining Points and Tangents into Parabolic Polygons

Image and geometry processing applications estimate the local geometry of objects using information localized at points. They usually consider information about the tangents as a side product of the points coordinates. This work proposes parabolic polygons as a model for discrete curves, which intrinsically combines points and tangents. This model is naturally affine invariant, which makes it particularly adapted to computer vision applications. As a direct application of this affine invariance, this paper introduces an affine curvature estimator that has a great potential to improve computer vision tasks such as matching and registering. As a proof-of-concept, this work also proposes an affine invariant curve reconstruction from point and tangent data.     

Shape recovery from turntable sequence using rim reconstruction

This paper makes use of both feature points and silhouettes to deliver fast 3D shape recovery. The algorithm exploits object silhouettes in two views to establish a 3D rim curve, which is defined with respect to the two frontier points arising from two views. The images of this 3D rim curve in the two views are matched using cross-correlation technique. A 3D planar rim curve is then reconstructed using point-based reconstruction method. A set of 3D rim curves enclosing the object can be obtained from an image sequence captured under circular motion. Silhouettes are further utilized to check for mismatched rim points. The proposed method solves the problem of reconstruction of concave object surface, which is usually left unresolved in general silhouette-based reconstruction methods. In addition, the property of the reconstructed 3D rim curves allows fast surface extraction. Experimental results with real data are presented.     

基于两幅图像的三维物体分层重构

文章研究在仅有两幅图像的条件下,如果摄像机内参数保持不变,如何利用场景中的结构信息实现空间物体的分层重构,研究表明,计算出无穷远平面的单应矩阵,可以仿射重构,计算出绝对二次曲线的像,就可以进一步实现欧氏重构。最后给出一个具体的实验,结果表明该算法是可行的。     

Discrete contours in multiple views: approximation and recognition

Recognition of discrete planar contours under similarity transformations has received a lot of attention but little work has been reported on recognizing them under more general transformations. Planar object boundaries undergo projective or affine transformations across multiple views. We present two methods to recognize discrete curves in this paper. The first method computes a piecewise parametric approximation of the discrete curve that is projectively invariant. A polygon approximation scheme and a piecewise conic approximation scheme are presented here. The second method computes an invariant sequence directly from the sequence of discrete points on the curve in a Fourier transform space. The sequence is shown to be identical up to a scale factor in all affine related views of the curve. We present the theory and demonstrate its applications to several problems including numeral recognition, aircraft recognition, and homography computation.     

Image Transformations and Blurring

Since cameras blur the incoming light during measurement, different images of the same surface do not contain the same information about that surface. Thus, in general, corresponding points in multiple views of a scene have different image intensities. While multiple-view geometry constrains the locations of corresponding points, it does not give relationships between the signals at corresponding locations. This paper offers an elementary treatment of these relationships. We first develop the notion of "ideal” and "real” images, corresponding to, respectively, the raw incoming light and the measured signal. This framework separates the filtering and geometric aspects of imaging. We then consider how to synthesize one view of a surface from another; if the transformation between the two views is affine, it emerges that this is possible if and only if the singular values of the affine matrix are positive. Next, we consider how to combine the information in several views of a surface into a single output image. By developing a new tool called "frequency segmentation,” we show how this can be done despite not knowing the blurring kernel.     

基于仿射迭代模型的特征点匹配算法

图像序列中的特征点匹配是计算机视觉中的一个基本问题,也是目标识别、图像检索以及3维重建等问题的基础。为了提高图像匹配的精度,提出了一种针对两幅图像的高精度特征点自动匹配算法。该算法首先分析并提出两幅图像中相应特征点的邻域窗口之间的单应映射可以用仿射变换模型来近似;然后通过快速的基于仿射变换模型的迭代优化方法,不仅估计并矫正了相应邻域窗口之间的透视畸变,同时还补偿了在特征点检测阶段对相应特征点的定位误差,从而使匹配结果达到子像素级精度;最后通过真实图像的实验以及与现有算法的比较结果表明,该算法不仅得到了更多的匹配关系,还提高了特征点匹配的精度。     

Uncalibrated reconstruction: an adaptation to structured light vision

Euclidean reconstruction from two uncalibrated stereoscopic views is achievable from the knowledge of geometrical constraints about the environment. Unfortunately, these constraints may be quite difficult to obtain. In this paper, we propose an approach based on structured lighting, which has the advantage of providing geometrical constraints independent of the scene geometry. Moreover, the use of structured light provides a unique solution to the tricky correspondence problem present in stereovision. The projection matrices are first computed by using a canonical representation, and a projective reconstruction is performed. Then, several constraints are generated from the image analysis and the projective reconstruction is upgraded into an Euclidean one—as we will demonstrate, it is assumed that the sensor behaviour is affine without loss of generality so that the constraints generation is simplified. The method provides our sensor with adaptive capabilities and permits to be used in the measurement of moving scenes such as dynamic visual inspection or mobile robot navigation. Experimental results obtained from both simulated and real data are presented.     

Relative affine structure: canonical model for 3D from 2D geometryand applications

We propose an affine framework for perspective views, captured by a single extremely simple equation based on a viewer-centered invariant we call relative affine structure. Via a number of corollaries of our main results we show that our framework unifies previous work-including Euclidean, projective and affine-in a natural and simple way, and introduces new, extremely simple algorithms for the tasks of reconstruction from multiple views, recognition by alignment, and certain image coding applications     

Self-Calibration Under the Cayley Framework

The Cayley framework here is meant to tackle the vision problems under the infinite Cayley transformation (ICT), its main advantage lies in its numerical stability. In this work, the stratified self-calibration under the Cayley framework is investigated. It is well known that the main difficulty of the stratified self-calibration in multiple view geometry is to upgrade a projective reconstruction to an affine one, in other words, to estimate the unknown 3-vector of the plane at infinity, called the normal vector. To our knowledge, without any prior knowledge about the scene or the camera motion, the only available constraint on a moving camera with constant intrinsic parameters is the well-known Modulus Constraint in the literature. Do other kinds of constraints exist? If yes, what they are? How could they be used? In this work, such questions will be systematically investigated under the Cayley framework. Our key contributions include: 1. The original projective expression of the ICT is simplified and a new projective expression is derived to make the upgrade easier from a projective reconstruction to a metric reconstruction. 2. The constraints on the normal vector are systematically investigated. For two views, two constraints on the normal vector are derived; one of them is the well-known modulus constraint, while the other is a new inequality constraint. There are only these two constraints for two views. For three views, besides the constraints for two views, two groups of new constraints are derived and each of them contains three constraints. In other words, there are 12 constraints in total for three views. 3. Based on our projective expression and these constraints, a stratified Cayley algorithm and a total Cayley algorithm are proposed for the metric reconstruction from images. It is experimentally shown that they both improve significantly the numerical stability of the classical algorithms. Compared with the global optimal algorithm under the infinite homography framework, the Cayley algorithms have comparable calibration accuracy, but substantially reduce the computational load.     

Three dimensional reconstruction of coronary arteries from two views

Geometric representation and measurements of localized lumen stenosis of coronary arteries are important considerations in the diagnosis of cardiovascular diseases. This discrete narrowing of the arteries typically impairs blood flow in regions of the heart, and can be present along the entire length of the artery. Three-dimensional (3-D) reconstruction of coronary arterial tree allows clinician to visualize vascular geometry. Three-dimensional representation of tree topology facilitates calculation of hemodynamic measurements to study myocardial infarction and stenosis. The 3-D arterial tree, computed from two views, can provide more information about the tree geometry than individual views. In this paper, a 3-step algorithm for 3-D reconstruction of arterial tree using two standard views is presented. The first step is a multi-resolution segmentation of the coronary vessels followed by medial-axis detection along the entire arterial tree for both views. In the second step, arterial trees from the two views are registered using medial-axis representation at the coarsest resolution level to obtain an initial 3-D reconstruction. This initial reconstruction at the coarsest level is then modified using 3-D geometrical a priori information. In the third step, the modified reconstruction is projected on the next higher-resolution segmented medial-axis representation and an updated reconstruction is obtained at the higher resolution. The process is iterated until the final 3-D reconstruction is obtained at the finest resolution level. Linear programming based constrained optimization method is used for registering two views at the coarse resolution. This is followed by a Tree-Search method for registering detailed branches at higher resolutions. The automated 3-D reconstruction method was evaluated on computer-simulated as well as human angiogram data. Results show that the automated 3-D reconstruction method provided good registration of computer-simulated data. On human angiogram data, the computed 3-D reconstruction matched well with manual registration.     

A new method for camera stratified self-calibration under circular motion

We consider the stratified self-calibration (affine and metric reconstruction) problem from images acquired with a camera with unchanging internal parameters undergoing circular motion. The general stratified method (modulus constraints) is known to fail with this motion. In this paper we give a novel constraint on the plane at infinity in projective reconstruction for circular motion, the constant inter-frame motion constraint on the plane at infinity between every two adjacent views and a fixed view of the motion sequences, by making use of the facts that in many commercial systems rotation angles are constant. An initial solution can be obtained by using the first three views of the sequence, and Stratified Iterative Particle Swarm Optimization (SIPSO) is proposed to get an accurate and robust solution when more views are at hand. Instead of using the traditional optimization algorithm as the last step to obtain an accurate solution, in this paper, the whole motion sequence information is exploited before computing the camera calibration matrix, this results in a more accurate and robust solution. Once the plane at infinity is identified, the calibration matrices of the camera and a metric reconstruction can be readily obtained. Experiments on both synthetic and real image sequence are given, showing the accuracy and robustness of the new algorithm.     

Conic reconstruction and correspondence from two views

Conics are widely accepted as one of the most fundamental image features together with points and line segments. The problem of space reconstruction and correspondence of two conics from two views is addressed in this paper. It is shown that there are two independent polynomial conditions on the corresponding pair of conics across two views, given the relative orientation of the two views. These two correspondence conditions are derived algebraically and one of them is shown to be fundamental in establishing the correspondences of conics. A unified closed-form solution is also developed for both projective reconstruction of conics in space from two uncalibrated camera views and metric reconstruction from two calibrated camera views. Experiments are conducted to demonstrate the discriminality of the correspondence conditions and the accuracy and stability of the reconstruction both for simulated and real images