线性确定无穷远平面的单应矩阵和摄象机自标定

引入了一种新的对无穷远平面的单应性矩阵(The infinite homography)的约束方程并 据此提出了一种新的摄象机线性自标定算法.与文献中已有的方法相比,该方法对摄象机的运 动要求不苛刻(如不要求摄象机的运动为正交运动),只须摄象机作一次平移运动和两次任意刚 体运动,就可线性唯一确定内参数.该方法主要优点在于:在确定无穷远平面的单应性矩阵的过 程中,不需要射影重构,也不需要有限远平面信息,唯一所需要的信息是图象极点,从而简化了 文献中现有的算法.另外同时给出了由极点确定(运动组)关于无穷远平面单应性矩阵的充分必 要条件.模拟实验和实际图象实验验证了该方法的正确性和可行性.     

基于正交平面的摄像机自定标

提出了一种基于正交平面摄像机定标的新算法。它利用场景中的正交平面,摄像机作五次以上的平移运动,根据每次运动关于平面的单应矩阵建立内参数的线性约束方程组,从而线性地确定内参数。与以往的定标方法相比,文章对摄像机的运动不苛刻,只需控制摄像机作平移运动,这在一般的实验平台上可以很容易地实现,并且线性地确定摄像机所有的五个内参数。模拟实验和真实图象实验表明,文章给出的方法在机器人视觉中具有一定的实用价值。     

Depth recovery and affine reconstruction under camera pure translation

This paper addresses the problems of depth recovery and affine reconstruction from two perspective images, which are generated by an uncalibrated translating camera. Firstly, we develop a new constraint that the homography for the plane, which is orthogonal to the optical axis, is determined only by the epipole and the plane's relative distance to the origin under camera pure translation. The algorithm of depth recovery is based on this new constraint, and it can successfully avoid the step of camera calibration. With the recovered depth, we show that affine reconstruction can be obtained readily. The proposed affine reconstruction does not need any control points, which were used to expand the affine coordinate system in existing method. Therefore, it could avoid the step of non-planarity verification as well as the errors from the control points. Error analysis is also presented to evaluate the uncertainty for the recovered depth value. Finally, we have tested the proposed algorithm with both simulated data and real image data. And the results show that the proposed algorithm is accurate and practical.     

Autocalibration of a projector-camera system

This paper presents a method for calibrating a projector-camera system that consists of multiple projectors (or multiple poses of a single projector), a camera, and a planar screen. We consider the problem of estimating the homography between the screen and the image plane of the camera or the screen-camera homography, in the case where there is no prior knowledge regarding the screen surface that enables the direct computation of the homography. It is assumed that the pose of each projector is unknown while its internal geometry is known. Subsequently, it is shown that the screen-camera homography can be determined from only the images projected by the projectors and then obtained by the camera, up to a transformation with four degrees of freedom. This transformation corresponds to arbitrariness in choosing a two-dimensional coordinate system on the screen surface and when this coordinate system is chosen in some manner, the screen-camera homography as well as the unknown poses of the projectors can be uniquely determined. A noniterative algorithm is presented, which computes the homography from three or more images. Several experimental results on synthetic as well as real images are shown to demonstrate the effectiveness of the method.     

Self-calibration of wireless cameras with restricted degrees of freedom

This paper presents an approach for the automatic calibration of low-cost cameras which are assumed to be restricted in their freedom of movement to either pan or tilt movements. Camera parameters, including focal length, principal point, lens distortion parameter and the angle and axis of rotation, can be recovered from a minimum set of two images of the camera, provided that the axis of rotation between the two images goes through the camera’s optical center and is parallel to either the vertical (panning) or horizontal (tilting) axis of the image. Previous methods for auto-calibration of cameras based on pure rotations fail to work in these two degenerate cases. In addition, our approach includes a modified RANdom SAmple Consensus (RANSAC) algorithm, as well as improved integration of the radial distortion coefficient in the computation of inter-image homographies. We show that these modifications are able to increase the overall efficiency, reliability and accuracy of the homography computation and calibration procedure using both synthetic and real image sequences.     

Straight lines have to be straight

Most algorithms in 3D computer vision rely on the pinhole camera model because of its simplicity, whereas video optics, especially low-cost wide-angle or fish-eye lenses, generate a lot of non-linear distortion which can be critical. To find the distortion parameters of a camera, we use the following fundamental property: a camera follows the pinhole model if and only if the projection of every line in space onto the camera is a line. Consequently, if we find the transformation on the video image so that every line in space is viewed in the transformed image as a line, then we know how to remove the distortion from the image. The algorithm consists of first doing edge extraction on a possibly distorted video sequence, then doing polygonal approximation with a large tolerance on these edges to extract possible lines from the sequence, and then finding the parameters of our distortion model that best transform these edges to segments. Results are presented on real video images, compared with distortion calibration obtained by a full camera calibration method which uses a calibration grid. Received: 27 December 1999 / Accepted: 8 November 2000     

Accurate Depth Dependent Lens Distortion Models: An Application to Planar View Scenarios

In order to calibrate cameras in an accurate manner, lens distortion models have to be included in the calibration procedure. Usually, the lens distortion models used in camera calibration depend on radial functions of image pixel coordinates. Such models are well-known, simple and can be estimated using just image information. However, these models do not take into account an important physical constraint of lens distortion phenomena, namely: the amount of lens distortion induced in an image point depends on the scene point depth with respect to the camera projection plane. In this paper we propose a new accurate depth dependent lens distortion model. To validate this approach, we apply the new lens distortion model to camera calibration in planar view scenarios (that is 3D scenarios where the objects of interest lie on a plane). We present promising experimental results on planar pattern images and on sport event scenarios. Nevertheless, although we emphasize the feasibility of the method for planar view scenarios, the proposed model is valid in general and can be used in any scenario where the point depth can be estimated.     

A versatile homography computation method based on two real points

In this paper, we present a novel 2D homography computation method based on two real points. The homography is thus decomposed into three parts. The two real points and their images can be utilized to compute the first and the last parts respectively, while other primitives (could be point(s), line(s) and conic) can be utilized to compute the middle part which is a hyperbolic similarity transformation. We introduce the proposed method in a 2D pattern with a conic and a coplanar line, and apply the method in various other geometric patterns. Subsequently, many plane-based vision tasks, such as camera calibration, pose estimation and metric rectification can be solved in a unified way as polynomial systems. The experiments with simulated and real data verify the correctness and the versatility of our algorithm.     

Augmented reality camera tracking with homographies

To realistically integrate 3D graphics into an unprepared environment, camera position must be estimated by tracking natural image features. We apply our technique to cases where feature positions in adjacent frames of an image sequence are related by a homography, or projective transformation. We describe this transformation's computation and demonstrate several applications. First, we use an augmented notice board to explain how a homography, between two images of a planar scene, completely determines the relative camera positions. Second, we show that the homography can also recover pure camera rotations, and we use this to develop an outdoor AR tracking system. Third, we use the system to measure head rotation and form a simple low-cost virtual reality (VR) tracking solution.     

结构光系统的自动标定

提出一种基于单应矩阵的摄像机自动标定算法。讨论摄像机焦距为恒定和任意变化两种情况下求解摄像机内参数的计算方法:论证空间平面诱导单应矩阵的性质,利用该性质不但能求出摄像机外参数,还可得到空间平面法向量和单应矩阵方程的比例因子。该算法在求解过程中不需要非线性迭代,可以直接获得解析解,实验表明该算法具有很好的准确性、普遍性。     

Semantic aware sport image resizing jointly using seam carving and warping

In content aware image resizing, saliency map or gradient is usually used to determine the important regions of images. But for sport images such as basketball and football images, these methods may falsely classify parts of court fields as unimportant regions, while parts of grandstands as important regions. Such results are not consistent with human perception. In this paper, a semantic aware image resizing approach is proposed. We extract the semantic information automatically. We segment the court fields as important regions and detect the boundary of court fields as the semantic edges. Considering the complementary characteristic of discrete image resizing approaches such as seam carving and continuous approaches such as warping, seam carving and warping are jointly used in our scheme. We define the Semantic Weight Function (SWF) based on the semantically important regions. Then semantic aware seam carving (SASC) is proposed based on the SWF. Next we define the Deformation of Semantic Edges (DSE) to assess the image deformation caused by seam carving. Finally seam carving and warping are joined using the DSE. We compare our approach with approaches like scaling, seam carving and semantic aware seam carving (SASC). Experimental results show that our approach preserves more semantically important regions with less deformation. Our approach also preserves the aspect ratio of key objects.     

单幅图像测量的一种新方法

本文方法表明,由一空间平面(参考平面)与其图像间的单应性矩阵(Homography)不仅 此参考平面上的距离可以测量,而且可以测量与此参考平面垂直的平面上的距离.同时,分别位 于两平面上的点间的距离也可以测量.这样就可以得到关于场景的更多的几何信息,此结果是在 前人的基础上又向前跨了一步.另外,本文提出一种新的基于平面单应性矩阵的摄像机标定方 法.模拟和真实图像试验均表明本文方法是可行的,并得到了令人满意的结果.     

1D camera geometry and its application to the self-calibration of circular motion sequences

This paper proposes a novel method for robustly recovering the camera geometry of an uncalibrated image sequence taken under circular motion. Under circular motion, all the camera centers lie on a circle and the mapping from the plane containing this circle to the horizon line observed in the image can be modelled as a 1D projection. A 2x2 homography is introduced in this paper to relate the projections of the camera centers in two 1D views. It is shown that the two imaged circular points of the motion plane and the rotation angle between the two views can be derived directly from such a homography. This way of recovering the imaged circular points and rotation angles is intrinsically a multiple view approach, as all the sequence geometry embedded in the epipoles is exploited in the estimation of the homography for each view pair. This results in a more robust method compared to those computing the rotation angles using adjacent views only. The proposed method has been applied to self-calibrate turntable sequences using either point features or silhouettes, and highly accurate results have been achieved.     

Camera calibration based on arbitrary parallelograms

Existing algorithms for camera calibration and metric reconstruction are not appropriate for image sets containing geometrically transformed images for which we cannot apply the camera constraints such as square or zero-skewed pixels. In this paper, we propose a framework to use scene constraints in the form of camera constraints. Our approach is based on image warping using images of parallelograms. We show that the warped image using parallelograms constrains the camera both intrinsically and extrinsically. Image warping converts the calibration problems of transformed images into the calibration problem with highly constrained cameras. In addition, it is possible to determine affine projection matrices from the images without explicit projective reconstruction. We introduce camera motion constraints of the warped image and a new parameterization of an infinite homography using the warping matrix. Combining the calibration and the affine reconstruction results in the fully metric reconstruction of scenes with geometrically transformed images. The feasibility of the proposed algorithm is tested with synthetic and real data. Finally, examples of metric reconstructions are shown from the geometrically transformed images obtained from the Internet.     

基于线对应的单应矩阵估计及其在视觉测量中的应用

单应矩阵估计在视觉测量、摄像机标定、三维重建等领域有重要的应用价值, 但是在具体应用中如何鲁棒、精确地估计单应矩阵仍是一个没有很好解决的问题. 在研究和实际应用中我们发现,直接线性方法在基于线对应的单应矩阵估计中会出现在某些特殊的摄像机姿态下误差较大的情况. 针对这一情况, 我们提出了一种基于线对应的归一化单应矩阵估计方法并将其应用到视觉测量中,即通过简单的归一化操作使测量矩阵元素的大小分布尽量均匀, 从而降低了测量矩阵的条件数, 提高了算法的鲁棒性, 同时又保持了直接线性方法简单、快速、易实现等优点. 模拟实验和真实图像实验均验证了该方法的有效性.     

Player trajectory reconstruction for tactical analysis

To increase the performance of sport team, the tactical analysis of team from game video is essential. Trajectories of the players are the most useful cues in a sport video for tactical analysis. In this paper, we propose a technique to reconstruct the trajectories of players from broadcast basketball videos. We first propose a mosaic based approach to detect the boundary lines of court. Then, the locations of players are determined by the integration of shape and color visual information. A layered graph is constructed for the detected players, which includes all possible trajectories. A dynamic programming based algorithm is applied to find the trajectory of each player. Finally, the trajectories of players are displayed on a standard basketball court model by a homography transformation. In contrast to related works, our approach exploits more spatio-temporal information in video. Experimental results show that the proposed approach works well and outperforms some existing technique.     

An efficient image-mosaicing method based on multifeature matching

Mosaicing is connecting two or more images and making a new wide area image with no visible seam-lines. Several algorithms have been proposed to construct mosaics from image sequence where the camera motion is more or less complex. Most of these methods are based either on the interest points matching or on theoretical corner models. This paper describes a fully automated image-mosaicing method based on the regions and the Harris points primitives. Indeed, in order to limit the search window of potential homologous points, for each point of interest, regions segmentation and matching steps are being performed. This enables us to improve the reliability and the robustness of the Harris points matching process by estimating the camera motion. The main originality of the proposed system resides in the preliminary manipulation of regions matching, thus making it possible to estimate the rotation, the translation and the scale factor between two successive images of the input sequence. This estimation allows an initial alignment of the images along with the framing of the interest points search window, and therefore reducing considerably the complexity of the interest points matching algorithm. Then, the resolution of a minimization problem, altogether considering the couples of matched-points, permits us to perform the homography. In order to improve the mosaic continuity around junctions, radiometric corrections are applied. The validity of the herewith described method is illustrated by being tested on several sequences of complex and challenging images captured from real-world indoor and outdoor scenes. These simulations proved the validity of the proposed method against camera motions, illumination variations, acquirement conditions, moving objects and image noise. To determine the importance of the regions matching stage in motion estimation, as well as for the framing of the search window associated to a point of interest, we compared the matching points results of this described method with those produced using the zero-mean normalized cross correlation score (without regions matching). We made this comparison in the case of a simple motion (without the presence of a rotation around optical axis and/or a scale factor), in the case of a rotation and in the general case of an homothety. For justifying the effectiveness of this method, we proposed an objective assessment by defining a reconstruction error.

Easy Calibration of a Multi-projector Display System

In this paper, we present a method for the geometric calibration of a multi-projector display system. The method is such that in order to calibrate the system, the user is only required to place the projectors and capture a single image of the images projected from them onto a planar screen using a hand-held camera. The problem to be solved is divided into the image registration for stitching different projector images into a single seamless image and the image rectification for making the image have the correct rectangular shape. The proposed method is characterized by simultaneously solving both of them from only a single image, which makes the calibration procedures easy. The method assumes an uncalibrated camera and partially calibrated projectors in which only focal lengths are unknown among the internal parameters. In the paper, we first prove the uniqueness of solutions to the problem, which was unclear in the previous studies, and then present a stable numerical algorithm for actually finding the solution. We present several experimental results for synthetic data, in which we show the relation between the calibration accuracy and several factors, and also present experimental results for real data, in which we demonstrate that the proposed method can calibrate a real system with sufficient accuracy for a number of layouts of the projectors.     

Multiview constraints on homographies

The image motion of a planar surface between two camera views is captured by a homography (a 2D projective transformation). The homography depends on the intrinsic and extrinsic camera parameters, as well as on the 3D plane parameters. While camera parameters vary across different views, the plane geometry remains the same. Based on this fact, we derive linear subspace constraints on the relative homographies of multiple (⩾ 2) planes across multiple views. The paper has three main contributions: 1) We show that the collection of all relative homographies (homologies) of a pair of planes across multiple views, spans a 4-dimensional linear subspace. 2) We show how this constraint can be extended to the case of multiple planes across multiple views. 3) We show that, for some restricted cases of camera motion, linear subspace constraints apply also to the set of homographies of a single plane across multiple views. All the results derived are true for uncalibrated cameras. The possible utility of these multiview constraints for improving homography estimation and for detecting nonrigid motions are also discussed     

Segment Based Camera Calibration

The basic idea of calibrating a camera system in previous approaches is to determine camera parmeters by using a set of known 3D points as calibration reference.In this paper,we present a method of camera calibration in whih camera parameters are determined by a set of 3D lines.A set of constraints is derived on camea parameters in terms of perspective line mapping.Form these constraints,the same perspective transformation matrix as that for point mapping can be computed linearly.The minimum number of calibration lines is 6.This result generalizes that of Liu,Huang and Faugeras^[12] for camera location determination in which at least 8 line correspondences are required for linear computation of camera location.Since line segments in an image can be located easily and more accurately than points,the use of lines as calibration reference tends to ease the computation in inage preprocessing and to improve calibration accuracy.Experimental results on the calibration along with stereo reconstruction are reported.