一种旋转平移运动模糊单帧图像恢复方法

成像传感器与目标景物之间有相当大的相对旋转角度和平移距离,经常会造成自然图像的运动模糊效应,为恢复旋转平移运动模糊图像,探讨了一种旋转平移运动模糊单帧图像恢复方法。     

Recovery of ego-motion using region alignment

A method for computing the 3D camera motion (the ego-motion) in a static scene is described, where initially a detected 2D motion between two frames is used to align corresponding image regions. We prove that such a 2D registration removes all effects of camera rotation, even for those image regions that remain misaligned. The resulting residual parallax displacement field between the two region-aligned images is an epipolar field centered at the FOE (Focus-of-Expansion). The 3D camera translation is recovered from the epipolar field. The 3D camera rotation is recovered from the computed 3D translation and the detected 2D motion. The decomposition of image motion into a 2D parametric motion and residual epipolar parallax displacements avoids many of the inherent ambiguities and instabilities associated with decomposing the image motion into its rotational and translational components, and hence makes the computation of ego-motion or 3D structure estimation more robust     

A review and evaluation of methods estimating ego-motion

If a visual observer moves through an environment, the patterns of light that impinge its retina vary leading to changes in sensed brightness. Spatial shifts of brightness patterns in the 2D image over time are called optic flow. In contrast to optic flow visual motion fields denote the displacement of 3D scene points projected onto the camera’s sensor surface. For translational and rotational movement through a rigid scene parametric models of visual motion fields have been defined. Besides ego-motion these models provide access to relative depth, and both ego-motion and depth information is useful for visual navigation.In the past 30 years methods for ego-motion estimation based on models of visual motion fields have been developed. In this review we identify five core optimization constraints which are used by 13 methods together with different optimization techniques.¹ In the literature methods for ego-motion estimation typically have been evaluated by using an error measure which tests only a specific ego-motion. Furthermore, most simulation studies used only a Gaussian noise model. Unlike, we test multiple types and instances of ego-motion. One type is a fixating ego-motion, another type is a curve-linear ego-motion. Based on simulations we study properties like statistical bias, consistency, variability of depths, and the robustness of the methods with respect to a Gaussian or outlier noise model. In order to achieve an improvement of estimates for noisy visual motion fields, part of the 13 methods are combined with techniques for robust estimation like m-functions or RANSAC. Furthermore, a realistic scenario of a stereo image sequence has been generated and used to evaluate methods of ego-motion estimation provided by estimated optic flow and depth information.     

Self-Calibration from Image Derivatives

This study investigates the problem of estimating camera calibration parameters from image motion fields induced by a rigidly moving camera with unknown parameters, where the image formation is modeled with a linear pinhole-camera model. The equations obtained show the flow to be separated into a component due to the translation and the calibration parameters and a component due to the rotation and the calibration parameters. A set of parameters encoding the latter component is linearly related to the flow, and from these parameters the calibration can be determined.However, as for discrete motion, in general it is not possible to decouple image measurements obtained from only two frames into translational and rotational components. Geometrically, the ambiguity takes the form of a part of the rotational component being parallel to the translational component, and thus the scene can be reconstructed only up to a projective transformation. In general, for full calibration at least four successive image frames are necessary, with the 3D rotation changing between the measurements.The geometric analysis gives rise to a direct self-calibration method that avoids computation of optical flow or point correspondences and uses only normal flow measurements. New constraints on the smoothness of the surfaces in view are formulated to relate structure and motion directly to image derivatives, and on the basis of these constraints the transformation of the viewing geometry between consecutive images is estimated. The calibration parameters are then estimated from the rotational components of several flow fields. As the proposed technique neither requires a special set up nor needs exact correspondence it is potentially useful for the calibration of active vision systems which have to acquire knowledge about their intrinsic parameters while they perform other tasks, or as a tool for analyzing image sequences in large video databases.     

Observability of 3D Motion

This paper examines the inherent difficulties in observing 3D rigid motion from image sequences. It does so without considering a particular estimator. Instead, it presents a statistical analysis of all the possible computational models which can be used for estimating 3D motion from an image sequence. These computational models are classified according to the mathematical constraints that they employ and the characteristics of the imaging sensor (restricted field of view and full field of view). Regarding the mathematical constraints, there exist two principles relating a sequence of images taken by a moving camera. One is the epipolar constraint, applied to motion fields, and the other the positive depth constraint, applied to normal flow fields. 3D motion estimation amounts to optimizing these constraints over the image. A statistical modeling of these constraints leads to functions which are studied with regard to their topographic structure, specifically as regards the errors in the 3D motion parameters at the places representing the minima of the functions. For conventional video cameras possessing a restricted field of view, the analysis shows that for algorithms in both classes which estimate all motion parameters simultaneously, the obtained solution has an error such that the projections of the translational and rotational errors on the image plane are perpendicular to each other. Furthermore, the estimated projection of the translation on the image lies on a line through the origin and the projection of the real translation. The situation is different for a camera with a full (360 degree) field of view (achieved by a panoramic sensor or by a system of conventional cameras). In this case, at the locations of the minima of the above two functions, either the translational or the rotational error becomes zero, while in the case of a restricted field of view both errors are non-zero. Although some ambiguities still remain in the full field of view case, the implication is that visual navigation tasks, such as visual servoing, involving 3D motion estimation are easier to solve by employing panoramic vision. Also, the analysis makes it possible to compare properties of algorithms that first estimate the translation and on the basis of the translational result estimate the rotation, algorithms that do the opposite, and algorithms that estimate all motion parameters simultaneously, thus providing a sound framework for the observability of 3D motion. Finally, the introduced framework points to new avenues for studying the stability of image-based servoing schemes.     

The Problem of Degeneracy in Structure and Motion Recovery from Uncalibrated Image Sequences

The aim of this work is the recovery of 3D structure and camera projection matrices for each frame of an uncalibrated image sequence. In order to achieve this, correspondences are required throughout the sequence. A significant and successful mechanism for automatically establishing these correspondences is by the use of geometric constraints arising from scene rigidity. However, problems arise with such geometry guided matching if general viewpoint and general structure are assumed whilst frames in the sequence and/or scene structure do not conform to these assumptions. Such cases are termed degenerate.In this paper we describe two important cases of degeneracy and their effects on geometry guided matching. The cases are a motion degeneracy where the camera does not translate between frames, and a structure degeneracy where the viewed scene structure is planar. The effects include the loss of correspondences due to under or over fitting of geometric models estimated from image data, leading to the failure of the tracking method. These degeneracies are not a theoretical curiosity, but commonly occur in real sequences where models are statistically estimated from image points with measurement error.We investigate two strategies for tackling such degeneracies: the first uses a statistical model selection test to identify when degeneracies occur: the second uses multiple motion models to overcome the degeneracies. The strategies are evaluated on real sequences varying in motion, scene type, and length from 13 to 120 frames.     

On the Geometry of Visual Correspondence

Image displacement fields—optical flow fields, stereo disparity fields, normal flow fields—due to rigid motion possess a global geometric structure which is independent of the scene in view. Motion vectors of certain lengths and directions are constrained to lie on the imaging surface at particular loci whose location and form depends solely on the 3D motion parameters. If optical flow fields or stereo disparity fields are considered, then equal vectors are shown to lie on conic sections. Similarly, for normal motion fields, equal vectors lie within regions whose boundaries also constitute conics. By studying various properties of these curves and regions and their relationships, a characterization of the structure of rigid motion fields is given. The goal of this paper is to introduce a concept underlying the global structure of image displacement fields. This concept gives rise to various constraints that could form the basis of algorithms for the recovery of visual information from multiple views.     

Using vanishing points for camera calibration

In this article a new method for the calibration of a vision system which consists of two (or more) cameras is presented. The proposed method, which uses simple properties of vanishing points, is divided into two steps. In the first step, the intrinsic parameters of each camera, that is, the focal length and the location of the intersection between the optical axis and the image plane, are recovered from a single image of a cube. In the second step, the extrinsic parameters of a pair of cameras, that is, the rotation matrix and the translation vector which describe the rigid motion between the coordinate systems fixed in the two cameras are estimated from an image stereo pair of a suitable planar pattern. Firstly, by matching the corresponding vanishing points in the two images the rotation matrix can be computed, then the translation vector is estimated by means of a simple triangulation. The robustness of the method against noise is discussed, and the conditions for optimal estimation of the rotation matrix are derived. Extensive experimentation shows that the precision that can be achieved with the proposed method is sufficient to efficiently perform machine vision tasks that require camera calibration, like depth from stereo and motion from image sequence.     

Exact two-image structure from motion

For two-image structure from motion, we present a simple, exact expression for a least-squares image-reprojection error for finite motion that depends only on the motion. Optimal estimates of the structure and motion can be computed by minimizing this expression just over the motion parameters. Also, we present a solution to the triangulation problem: an exact, explicit expression for the optimal structure estimate given the motion. We identify a new ambiguity in recovering the structure for known motion. We study the exact error's properties experimentally and demonstrate that it often has several local minima for forward or backward motion estimates. Our experiments also show that the "reflected" local minimum of Oliensis (2001) and Soatto et al. (1998) occurs for large translational motions. Our exact results assume that the camera is calibrated and use a least-squares image-reprojection error that applies most naturally to a spherical imaging surface. We approximately extend our approach to the standard least-squares error in the image plane and uncalibrated cameras. We present an improved version of the Sampson error which gives better results experimentally.     

Direct recovery of planar-parallax from multiple frames

We present an algorithm that estimates dense planar-parallax motion from multiple uncalibrated views of a 3D scene. This generalizes the "plane+parallax" recovery methods to more than two frames. The parallax motion of pixels across multiple frames (relative to a planar surface) is related to the 3D scene structure and the camera epipoles. The parallax field, the epipoles, and the 3D scene structure are estimated directly from image brightness variations across multiple frames, without precomputing correspondences.     

Generic self-calibration of central cameras

We consider the self-calibration problem for a generic imaging model that assigns projection rays to pixels without a parametric mapping. We consider the central variant of this model, which encompasses all camera models with a single effective viewpoint. Self-calibration refers to calibrating a camera’s projection rays, purely from matches between images, i.e. without knowledge about the scene such as using a calibration grid. In order to do this we consider specific camera motions, concretely, pure translations and rotations, although without the knowledge of rotation and translation parameters (rotation angles, axis of rotation, translation vector). Knowledge of the type of motion, together with image matches, gives geometric constraints on the projection rays. We show for example that with translational motions alone, self-calibration can already be performed, but only up to an affine transformation of the set of projection rays. We then propose algorithms for full metric self-calibration, that use rotational and translational motions or just rotational motions.     

Motion and structure from two perspective views: algorithms, erroranalysis, and error estimation

Deals with estimating motion parameters and the structure of the scene from point (or feature) correspondences between two perspective views. An algorithm is presented that gives a closed-form solution for motion parameters and the structure of the scene. The algorithm utilizes redundancy in the data to obtain more reliable estimates in the presence of noise. An approach is introduced to estimating the errors in the motion parameters computed by the algorithm. Specifically, standard deviation of the error is estimated in terms of the variance of the errors in the image coordinates of the corresponding points. The estimated errors indicate the reliability of the solution as well as any degeneracy or near degeneracy that causes the failure of the motion estimation algorithm. The presented approach to error estimation applies to a wide variety of problems that involve least-squares optimization or pseudoinverse. Finally the relationships between errors and the parameters of motion and imaging system are analyzed. The results of the analysis show, among other things, that the errors are very sensitive to the translation direction and the range of field view. Simulations are conducted to demonstrate the performance of the algorithms and error estimation as well as the relationships between the errors and the parameters of motion and imaging systems. The algorithms are tested on images of real-world scenes with point of correspondences computed automatically     

采用改进的几何算法快速估计图像旋转角度

不规则的旋转运动会明显的降低图像序列的质量,旋转运动估计是实现电子图像稳定的关键技术.采用仿射变换模型可以有效地估计图像二维运动参数,但是求解最小二乘解的计算量太大.提出了一种简单的估计图像旋转角度的方法.利用匹配块相对于旋转中心位移矢量对称的特性,消除平移运动引起的运动矢量,通过简单的几何计算即町获得旋转参数,进一步,通过合理设置阈值,降低局部运动估计误差对全局结果的影响.理论分析和仿真结果表明,在帧间图像旋转角度小于10°的条件下,应用方法能够简单有效的估计出旋转角度.     

Motion parameter estimation from global flow field data

Presented are two methods for the determination of the parameters of motion of a sensor, given the vector flow field induced by an imaging system governed by a perspective transformation of a rigid scene. Both algorithms integrate global data to determine motion parameters. The first (the flow circulation algorithm) determines the rotational parameters. The second (the FOE search algorithm) determines the translational parameters of the motion independently of the first algorithm. Several methods for determining when the function has the appropriate form are suggested. One method involves filtering the function by a collection of circular-surround zero-mean receptive fields. The other methods project the function onto a linear space of quadratic polynomials and measures the distance between the two functions. The error function for the first two methods is a quadratic polynomial of the candidate position, yielding a very rapid search strategy     

The feasibility of motion and structure from noisy time-varying image velocity information

This research addresses the problem of noise sensitivity inherent in motion and structure algorithms. The motion and structure paradigm is a two-step process. First, we measure image velocities and, perhaps, their spatial and temporal derivatives, are obtained from time-varying image intensity data and second, we use these data to compute the motion of a moving monocular observer in a stationary environment under perspective projection, relative to a single 3-D planar surface. The first contribution of this article is an algorithm that uses time-varying image velocity information to compute the observer's translation and rotation and the normalized surface gradient of the 3-D planar surface. The use of time-varying image velocity information is an important tool in obtaining a more robust motion and structure calculation. The second contribution of this article is an extensive error analysis of the motion and structure problem. Any motion and structure algorithm that uses image velocity information as its input should exhibit error sensitivity behavior compatible with the results reported here. We perform an average and worst case error analysis for four types of image velocity information: full and normal image velocities and full and normal sets of image velocity and its derivatives. (These derivatives are simply the coefficients of a truncated Taylor series expansion about some point in space and time.) The main issues we address here are: just how sensitive is a motion and structure computation in the presence of noisy input, or alternately, how accurate must our image velocity information be, how much and what type of input data is needed, and under what circumstances is motion and structure feasible? That is, when can we be sure that a motion and structure computation will produce usable results? We base our answers on a numerical error analysis we conduct for a large number of motions.     

Qualitative egomotion

Due to the aperture problem, the only motion measurement in images, whose computation does not require any assumptions about the scene in view, is normal flow—the projection of image motion on the gradient direction. In this paper we show how a monocular observer can estimate its 3D motion relative to the scene by using normal flow measurements in a global and qualitative way. The problem is addressed through a search technique. By checking constraints imposed by 3D motion parameters on the normal flow field, the possible space of solutions is gradually reduced. In the four modules that comprise the solution, constraints of increasing restriction are considered, culminating in testing every single normal flow value for its consistency with a set of motion parameters. The fact that motion is rigid defines geometric relations between certain values of the normal flow field. The selected values form patterns in the image plane that are dependent on only some of the motion parameters. These patterns, which are determined by the signs of the normal flow values, are searched for in order to find the axes of translation and rotation. The third rotational component is computed from normal flow vectors that are only due to rotational motion. Finally, by looking at the complete data set, all solutions that cannot give rise to the given normal flow field are discarded from the solution space.Research supported in part by NSF (Grant IRI-90-57934), ONR (Contract N00014-93-1-0257) and ARPA (Order No. 8459).     

基于图象差的平面大范围视觉伺服控制

为解决大范围偏差的控制问题,将期望图象按给定的角度间隔旋转,离线生成一系列子期望图象。比较实时采集图象与期望子图象间的差异程序可获取目标绕重心的旋转运动参数。纵使图象求重心方法给出的平动参数,实现了在大范围偏差时迅速将摄象机调整到期望位姿。在期望位姿附近结合直接图象反馈方式,实现了基于图象差的平面大范围视觉伺服控制。     

Quantitative planar region detection

This paper presents a means of segmenting planar regions from two views of a scene using point correspondences. The initial selection of groups of coplanar points is performed on the basis of conservation of two five point projective invariants (groups for which this invariant is conserved are assumed to be coplanar). The five point correspondences are used to estimate a projectivity which is used to predict the change in position of other points assuming they lie on the same plane as the original four. The variance in any points new position is then used to define a distance threshold between actual and predicted position which is used as a coplanarity test to find extended planar regions. If two distinct planar regions can be found then a novel motion direction estimator suggests itself. The projection of the line of intersection of two planes in an image may also be recovered. An analytical error model is derived which relates image uncertainty in a corner's position to genuine perpendicular height of a point above a given plane in the world. The model may be used for example to predict the performance of given stereo ground plane prediction system or a monocular drivable region detection system on and AGV. The model may also be used in reverse to determine the camera resolution required if a vehicle in motion is to resolve obstacles of a given height a given distance from it.     

一种机器人手眼关系自标定方法

设计了一种基于场景中单个景物点的机器人手眼关系标定方法．精确控制机械手末端执行器做 5 次以上平移运动和2 次以上旋转运动,摄像机对场景中的单个景物点进行成像．通过景物点的视差及深度 值反映摄像机的运动,建立机械手末端执行器与摄像机两坐标系之间相对位置的约束方程组,线性求得摄像 机内参数及手眼关系．标定过程中只需提取场景中的一个景物点,无需匹配,无需正交运动,对机械手的运 动控制操作方便、算法实现简洁．模拟数据实验与真实图像数据实验结果表明该方法可行、有效．     

Estimating the kinematics and structure of a rigid object from asequence of monocular images

The problem considered involves the use of a sequence of noisy monocular images of a three-dimensional moving object to estimate both its structure and kinematics. The object is assumed to be rigid, and its motion is assumed to be smooth. A set of object match points is assumed to be available, consisting of fixed features on the object, the image plane coordinates of which have been extracted from successive images in the sequence. Structure is defined as the 3-D positions of these object feature points, relative to each other. Rotational motion occurs about the origin of an object-centered coordinate system, while translational motion is that of the origin of this coordinate system. In this work, which is a continuation of the research done by the authors and reported previously (ibid., vol.PAMI-8, p.90-9, Jan. 1986), results of an experiment with real imagery are presented, involving estimation of 28 unknown translational, rotational, and structural parameters, based on 12 images with seven feature points