Self-Calibration Under the Cayley Framework

The Cayley framework here is meant to tackle the vision problems under the infinite Cayley transformation (ICT), its main advantage lies in its numerical stability. In this work, the stratified self-calibration under the Cayley framework is investigated. It is well known that the main difficulty of the stratified self-calibration in multiple view geometry is to upgrade a projective reconstruction to an affine one, in other words, to estimate the unknown 3-vector of the plane at infinity, called the normal vector. To our knowledge, without any prior knowledge about the scene or the camera motion, the only available constraint on a moving camera with constant intrinsic parameters is the well-known Modulus Constraint in the literature. Do other kinds of constraints exist? If yes, what they are? How could they be used? In this work, such questions will be systematically investigated under the Cayley framework. Our key contributions include: 1. The original projective expression of the ICT is simplified and a new projective expression is derived to make the upgrade easier from a projective reconstruction to a metric reconstruction. 2. The constraints on the normal vector are systematically investigated. For two views, two constraints on the normal vector are derived; one of them is the well-known modulus constraint, while the other is a new inequality constraint. There are only these two constraints for two views. For three views, besides the constraints for two views, two groups of new constraints are derived and each of them contains three constraints. In other words, there are 12 constraints in total for three views. 3. Based on our projective expression and these constraints, a stratified Cayley algorithm and a total Cayley algorithm are proposed for the metric reconstruction from images. It is experimentally shown that they both improve significantly the numerical stability of the classical algorithms. Compared with the global optimal algorithm under the infinite homography framework, the Cayley algorithms have comparable calibration accuracy, but substantially reduce the computational load.     

Self-calibration from one circular motion sequence and two images

This paper describes a new method for self-calibration of camera with constant internal parameters under circular motion, using one sequence and two images captured with different camera orientations. Unlike the previous method, in which three circular motion sequences are needed with known motion, the new method computes the rotation angles and the projective reconstructions of the sequence and the images with circular constraint enforced, which is called a circular projective reconstruction, using a factorization-based method. It is then shown that the images of the circular points of each circular projective reconstruction can be readily obtained. Subsequently, the image of the absolute conic and the calibration matrix of the camera can be determined. Experiments on both synthetic and real image sequence are given, showing the accuracy and robustness of the new algorithm.     

Stratified self-calibration with the modulus constraint

In computer vision and especially for 3D reconstruction, one of the key issues is the retrieval of the calibration parameters of the camera. These are needed to obtain metric information about the scene from the camera. Often these parameters are obtained through cumbersome calibration procedures. There is a way to avoid explicit calibration of the camera. Self-calibration is based on finding the set of calibration parameters which satisfy some constraints (e.g., constant calibration parameters). Several techniques have been proposed but it often proved difficult to reach a metric calibration at once. Therefore, in the paper, a stratified approach is proposed, which goes from projective through affine to metric. The key concept to achieve this is the modulus constraint. It allows retrieval of the affine calibration for constant intrinsic parameters. It is also suited for use in conjunction with scene knowledge. In addition, if the affine calibration is known, it can also be used to cope with a changing focal length     

Multi-stage 3D reconstruction under circular motion

In this paper, we propose a new method for 3D reconstruction from an image sequence captured by a camera with constant intrinsic parameters undergoing circular motion. We introduce a method, called circular projective reconstruction, for enforcing the circular constraint in a factorization-based projective reconstruction. To deal with the missing data problem, our method uses a multi-stage approach to reconstructing the objects and cameras, which first computes a circular projective reconstruction of a sub-sequence and then extends the reconstruction to the complete sequence. Camera matrix, rotation angles, and 3D structure are computed iteratively in a way that the 2D reprojection error is minimized. The algorithm is evaluated using real image sequences.     

1D camera geometry and its application to the self-calibration of circular motion sequences

This paper proposes a novel method for robustly recovering the camera geometry of an uncalibrated image sequence taken under circular motion. Under circular motion, all the camera centers lie on a circle and the mapping from the plane containing this circle to the horizon line observed in the image can be modelled as a 1D projection. A 2x2 homography is introduced in this paper to relate the projections of the camera centers in two 1D views. It is shown that the two imaged circular points of the motion plane and the rotation angle between the two views can be derived directly from such a homography. This way of recovering the imaged circular points and rotation angles is intrinsically a multiple view approach, as all the sequence geometry embedded in the epipoles is exploited in the estimation of the homography for each view pair. This results in a more robust method compared to those computing the rotation angles using adjacent views only. The proposed method has been applied to self-calibrate turntable sequences using either point features or silhouettes, and highly accurate results have been achieved.     

Self-Calibration of Turntable Sequences from Silhouettes

This paper addresses the problem of recovering both the intrinsic and extrinsic parameters of a camera from the silhouettes of an object in a turntable sequence. Previous silhouette-based approaches have exploited correspondences induced by epipolar tangents to estimate the image invariants under turntable motion and achieved a weak calibration of the cameras. It is known that the fundamental matrix relating any two views in a turntable sequence can be expressed explicitly in terms of the image invariants, the rotation angle, and a fixed scalar. It will be shown that the imaged circular points for the turntable plane can also be formulated in terms of the same image invariants and fixed scalar. This allows the imaged circular points to be recovered directly from the estimated image invariants, and provide constraints for the estimation of the imaged absolute conic. The camera calibration matrix can thus be recovered. A robust method for estimating the fixed scalar from image triplets is introduced, and a method for recovering the rotation angles using the estimated imaged circular points and epipoles is presented. Using the estimated camera intrinsics and extrinsics, a Euclidean reconstruction can be obtained. Experimental results on real data sequences are presented, which demonstrate the high precision achieved by the proposed method.     

Efficient three-dimensional metric object modeling from uncalibrated image sequences

This paper presents a scheme that addresses the practical issues associated with producing a geometric model of a scene using a passive sensing technique. The proposed image-based scheme comprises a recursive structure recovery method and a recursive surface reconstruction technique. The former method employs a robust corner-tracking algorithm that copes with the appearance and disappearance of features and a corner-based structure and motion estimation algorithm that handles the inclusion and expiration of features. The novel formulation and dual extended Kalman filter computational framework of the estimation algorithm provide an efficient approach to metric structure recovery that does not require any prior knowledge about the camera or scene. The newly developed surface reconstruction technique employs a visibility constraint to iteratively refine and ultimately yield a triangulated surface that envelops the recovered scene structure and can produce views consistent with those of the original image sequence. Results on simulated data and synthetic and real imagery illustrate that the proposed scheme is robust, accurate, and has good numerical stability, even when features are repeatedly absent or their image locations are affected by extreme levels of noise.     

A New Linear Method for Camera Self-Calibration with Planar Motion

We consider the self-calibration (affine and metric reconstruction) problem from images acquired with a camera with unchanging internal parameters undergoing planar motion. The general self-calibration methods (modulus constraint, Kruppa equations) are known to fail with this camera motion. In this paper we give two novel linear constraints on the coordinates of the plane at infinity in a projective reconstruction for any camera motion. In the planar case, we show that the two constraints are equivalent and easy to compute, giving us a linear version of the quartic modulus constraint. Using this fact, we present a new linear method to solve the self-calibration problem with planar motion of the camera from three or more images. This work was partly supported by project BFM2003-02914 from the Ministerio de Ciencia y Tecnología (Spain). Ferran Espuny received the MSc in Mathematics in 2002 from the Universitat de Barcelona, Spain. He is currently a PhD student and associate professor in the Departament d’àlgebra i Geometria at Universitat de Barcelona, Spain. His research, supervised by Dr. José Ignacio Burgos Gil, is focussed on self-calibration and critical motions for both pinhole and generic camera models.     

基于仿射点对应的分层重构

提出了一种基于仿射点对应的分层重构方法,所谓仿射点对应是指相差一个仿射变换的两个空间点集的图像对应．该方法主要分为以下三个步骤：首先,从点对应计算准仿射重构;然后,由仿射点对应的准仿射重构建立一个三维射影变换,并利用这个射影变换的特征向量来确定无穷远平面,从而得到仿射重构;最后,从仿射重构所获得的无穷远平面单应矩阵标定摄像机内参数,进而得到度量重构．在上述三个步骤中,第二个步骤是最关键的,即如何确定对应于无穷远平面的特征向量,这也是该文的新思想和主要贡献所在．仿真和真实图像实验均表明,该文的方法是有效的,并且有很好的鲁棒性．     

线性确定无穷远平面的单应矩阵和摄象机自标定

引入了一种新的对无穷远平面的单应性矩阵(The infinite homography)的约束方程并 据此提出了一种新的摄象机线性自标定算法.与文献中已有的方法相比,该方法对摄象机的运 动要求不苛刻(如不要求摄象机的运动为正交运动),只须摄象机作一次平移运动和两次任意刚 体运动,就可线性唯一确定内参数.该方法主要优点在于:在确定无穷远平面的单应性矩阵的过 程中,不需要射影重构,也不需要有限远平面信息,唯一所需要的信息是图象极点,从而简化了 文献中现有的算法.另外同时给出了由极点确定(运动组)关于无穷远平面单应性矩阵的充分必 要条件.模拟实验和实际图象实验验证了该方法的正确性和可行性.     

Camera Autocalibration and Horopter Curves

We describe a new algorithm for the obtainment of the affine and Euclidean calibration of a camera under general motion. The algorithm exploits the relationships of the horopter curves associated to each pair of cameras with the plane at infinity and the absolute conic. Using these properties we define cost functions whose minimization by means of general purpose techniques provides the required calibration. The experiments show the good convergence properties, computational efficiency and robust performance of the new techniques.     

Calibration-free augmented reality in perspective

This paper deals with video-based augmented reality and proposes an algorithm for augmenting a real video sequence with views of graphics objects without metric calibration of the video camera by representing the motion of the video camera in projective space. A virtual camera, by which views of graphics objects are generated, is attached to a real camera by specifying image locations of the world coordinate system of the virtual world. The virtual camera is decomposed into calibration and motion components in order to make full use of graphics tools. The projective motion of the real camera recovered from image matches has the function of transferring the virtual camera and makes the virtual camera move according to the motion of the real camera. The virtual camera also follows the change of the internal parameters of the real camera. This paper shows the theoretical and experimental results of our application of nonmetric vision to augmented reality     

Uncalibrated Motion Capture Exploiting Articulated Structure Constraints

We present an algorithm for 3D reconstruction of dynamic articulated structures, such as humans, from uncalibrated multiple views. The reconstruction exploits constraints associated with a dynamic articulated structure, specifically the conservation over time of length between rotational joints. These constraints admit reconstruction of metric structure from at least two different images in each of two uncalibrated parallel projection cameras. As a by product, the calibration of the cameras can also be computed. The algorithm is based on a stratified approach, starting with affine reconstruction from factorization, followed by rectification to metric structure using the articulated structure constraints. The exploitation of these specific constraints admits reconstruction and self-calibration with fewer feature points and views compared to standard self-calibration. The method is extended to pairs of cameras that are zooming, where calibration of the cameras allows compensation for the changing scale factor in a scaled orthographic camera. Results are presented in the form of stick figures and animated 3D reconstructions using pairs of sequences from broadcast television. The technique shows promise as a means of creating 3D animations of dynamic activities such as sports events.     

Multiple motion scene reconstruction with uncalibrated cameras

In this paper, we describe a reconstruction method for multiple motion scenes, which are scenes containing multiple moving objects, from uncalibrated views. Assuming that the objects are moving with constant velocities, the method recovers the scene structure, the trajectories of the moving objects, the camera motion, and the camera intrinsic parameters (except skews) simultaneously. We focus on the case where the cameras have unknown and varying focal lengths while the other intrinsic parameters are known. The number of the moving objects is automatically detected without prior motion segmentation. The method is based on a unified geometrical representation of the static scene and the moving objects. It first performs a projective reconstruction using a bilinear factorization algorithm and, then, converts the projective solution to a Euclidean one by enforcing metric constraints. Experimental results on synthetic and real images are presented.     

A new easy camera calibration technique based on circular points

Inspired by Zhang's work on flexible calibration technique, a new easy technique for calibrating a camera based on circular points is proposed. The proposed technique only requires the camera to observe a newly designed planar calibration pattern (referred to as the model plane hereinafter) which includes a circle and a pencil of lines passing through the circle's center, at a few (at least three) different unknown orientations, then all the five intrinsic parameters can be determined linearly. The main advantage of our new technique is that it needs to know neither any metric measurement on the model plane, nor the correspondences between points on the model plane and image ones, hence the whole calibration process becomes extremely simple. The proposed technique is particularly useful for those people who are not familiar with computer vision. Experiments with simulated data as well as with real images show that our new technique is robust and accurate.     

基于平面与直线的仿射重建

首先给出了无穷远平面的单应矩阵以及仿射重建算法，然后从数学上严格证明了下述命题：在变参数模型下，如果场景中含有一张平面和一对平行直线，或者场景中含有两张平行平面，则从两个平移视点下的图像均可以线性地对场景进行仿射重建；文章同时指出：如果场景中包含一对平行平面和一对平行直线，则从两个一般运动视点也可以线性地重建场景的仿射几何．大量的模拟和真实图像实验表明，该线性仿射重建算法是正确的，同时具有较高的重建精度和鲁棒性．     

Non-rigid metric reconstruction from perspective cameras

The metric reconstruction of a non-rigid object viewed by a generic camera poses new challenges since current approaches for Structure from Motion assume the rigidity constraint of a shape as an essential condition. In this work, we focus on the estimation of the 3-D Euclidean shape and motion of a non-rigid shape observed by a perspective camera. In such case deformation and perspective effects are difficult to decouple – the parametrization of the 3-D non-rigid body may mistakenly account for the perspective distortion. Our method relies on the fact that it is often a reasonable assumption that some of the points on the object’s surface are deforming throughout the sequence while others remain rigid. Thus, relying on the rigidity constraints of a subset of rigid points, we estimate the perspective to metric upgrade transformation. First, we use an automatic segmentation algorithm to identify the set of rigid points. These are then used to estimate the internal camera calibration parameters and the overall rigid motion. Finally, we formulate the problem of non-rigid shape and motion estimation as a non-linear optimization where the objective function to be minimized is the image reprojection error. The prior information that some of the points in the object are rigid can also be added as a constraint to the non-linear minimization scheme in order to avoid ambiguous configurations. We perform experiments on different synthetic and real data sets which show that even when using a minimal set of rigid points and when varying the intrinsic camera parameters it is possible to obtain reliable metric information.     

Camera calibration based on arbitrary parallelograms

Existing algorithms for camera calibration and metric reconstruction are not appropriate for image sets containing geometrically transformed images for which we cannot apply the camera constraints such as square or zero-skewed pixels. In this paper, we propose a framework to use scene constraints in the form of camera constraints. Our approach is based on image warping using images of parallelograms. We show that the warped image using parallelograms constrains the camera both intrinsically and extrinsically. Image warping converts the calibration problems of transformed images into the calibration problem with highly constrained cameras. In addition, it is possible to determine affine projection matrices from the images without explicit projective reconstruction. We introduce camera motion constraints of the warped image and a new parameterization of an infinite homography using the warping matrix. Combining the calibration and the affine reconstruction results in the fully metric reconstruction of scenes with geometrically transformed images. The feasibility of the proposed algorithm is tested with synthetic and real data. Finally, examples of metric reconstructions are shown from the geometrically transformed images obtained from the Internet.     

Euclidean upgrading from segment lengths: DLT-like algorithm and its variants

In this paper, how to calibrate a fixed multi-camera system and simultaneously achieve a Euclidean reconstruction from a set of segments is addressed. It is well known that only a projective reconstruction could be achieved without any prior information. Here, the known segment lengths are exploited to upgrade the projective reconstruction to a Euclidean reconstruction and simultaneously calibrate the intrinsic and extrinsic camera parameters. At first, a DLT(Direct Linear Transformation)-like algorithm for the Euclidean upgrading from segment lengths is derived in a very simple way. Although the intermediate results in the DLT-like algorithm are essentially equivalent to the quadric of segments (QoS), the DLT-like algorithm is of higher accuracy than the existing linear algorithms derived from the QoS because of a more accurate way to extract the plane at infinity from the intermediate results. Then, to further improve the accuracy of Euclidean upgrading, two weighted DLT-like algorithms are presented by weighting the linear constraint equations in the original DLT-like algorithm. Finally, using the results of these linear algorithms as the initial values, a new weighted nonlinear algorithm for Euclidean upgrading is explored to recover the Euclidean structure more accurately. Extensive experimental results on both the synthetic data and the real image data demonstrate the effectiveness of our proposed algorithms in Euclidean upgrading and multi-camera calibration.     

Camera calibration using identical objects

This paper describes a method for camera calibration using identical products. In this paper, we postulate an imaginative rigid motion between any two identical products, and the imaginative rigid motion could offer a pair of circular points. As is known, three pairs of projections of the circular points are needed to result in the closed-form solution for calibration. In our method, we obtain three pairs of projections of the circular points from only two images of three identical products, or three images of two identical products. When only two identical products are utilized, our method is almost the dual of the stereo calibration from rigid motions. A direct approach is taken here instead of the two-step process in stereo calibration. Furthermore, a better projective reconstruction could be performed from the estimation of the camera parameters to avoid the dominant projective-to-affine error in the stereo calibration. Finally, we conduct a nonlinear refinement based on the maximum likelihood estimation. The experimental results from synthetic data and real data prove our method convenient and robust to noise.