Combining Support Vector Machines and simulated annealing for stereovision matching with fish eye lenses in forest environments

We present a novel strategy for computing disparity maps from omni-directional stereo images obtained with fish-eye lenses in forest environments. At a first segmentation stage, the method identifies textures of interest to be either matched or discarded. Two of them are identified by applying the powerful Support Vector Machines approach. At a second stage, a stereovision matching process is designed based on the application of four stereovision matching constraints: epipolarity, similarity, uniqueness and smoothness. The epipolarity guides the process. The similarity and uniqueness are mapped once again through the Support Vector Machines, but under a different way to the previous case; after this an initial disparity map is obtained. This map is later filtered by applying the Discrete Simulated Annealing framework where the smoothness constraint is conveniently mapped. The combination of the segmentation and stereovision matching approaches makes the main contribution. The method is compared against the usage of simple features and combined similarity matching strategies.     

Fuzzy Cognitive Maps for stereovision matching

This paper outlines a method for solving the stereovision matching problem using edge segments as the primitives. In stereovision matching the following constraints are commonly used: epipolar, similarity, smoothness, ordering and uniqueness. We propose a new matching strategy under a fuzzy context in which such constraints are mapped. The fuzzy context integrates both Fuzzy Clustering and Fuzzy Cognitive Maps. With such purpose a network of concepts (nodes) is designed, each concept represents a pair of primitives to be matched. Each concept has associated a fuzzy value which determines the degree of the correspondence. The goal is to achieve high performance in terms of correct matches. The main findings of this paper are reflected in the use of the fuzzy context that allows building the network of concepts where the matching constraints are mapped. Initially, each concept value is loaded via the Fuzzy Clustering and then updated by the Fuzzy Cognitive Maps framework. This updating is achieved through the influence of the remainder neighboring concepts until a good global matching solution is achieved. Under this fuzzy approach we gain quantitative and qualitative matching correspondences. This method works as a relaxation matching approach and its performance is illustrated by comparative analysis against some existing global matching methods.     

非标定图像的最优匹配方法

该文将特征匹配和极线几何(epipolargeometry)估计有机地结合起来,给出了一种基于组合优化的非标定图像鲁棒匹配方法。通过灰度互相关计算得到初始候选匹配,然后使用该文提出的全局极线约束和局部视差约束代价函数,利用确定性退火方法同时估计匹配关系和基础矩阵。实验结果表明,此算法具有良好的鲁棒性,能够得到接近全局最优的匹配结果。     

基于编码结构光和外极线约束的自由曲面三维轮廓术

提出了一种新的基于编码结构光和外极线约束的自由曲面立体视觉测量方法,这种方法有效解决了立体视觉中的图像匹配问题;外极线约束将可能的候选点限制在直线分布,引入编码光用来确定光栅条纹的级次;这种方法极大地减小了图像匹配的运算量,同时减小了错误匹配的概率;实验表明,采用编码光栅投影视觉测量能够高效准确地测量自由曲面的三维轮廓.     

A 1D approach to correlation-based stereo matching

In stereovision, indices allowing pixels of the left and right images to be matched are basically one-dimensional features of the epipolar lines. In some situations, these features are not significant or cannot be extracted from the single epipolar line. Therefore, many techniques use 2D neighbourhoods to increase the available information. In this paper, we discuss the systematic use of 2D neighbourhoods for stereo matching. We propose an alternative approach to stereo matching using multiple 1D correlation windows, which yields a semi-dense disparity map and an associated confidence map. A particular technique derived from this approach — using fuzzy filtering and a basic decision rule — is compared to about 80 other methods on the Middlebury image datasets [1]. Results are first presented in the framework of the Middlebury website, then on the Receiver Operating Characteristics (ROC) evaluation [2] and, finally, on stereo image pairs of slanted surfaces. We show that a 1D correlation window is sufficient to provide correct matchings in most cases.     

一种基于局部最大熵的特征匹配算法

传统的基于灰度的匹配算法抗噪声能力和抗局部几何变形能力较差，通过图像熵变换，提出了一种新颖的基于局部最大熵的特征匹配算法；通过局部特征点所在区域的相关匹配，获得具有最大可信度的匹配结果。由于匹配只是在特征点之间进行，且在匹配过程中引入外极线和一致性约束条件，从而大大降低了计算消耗和误匹配率，获得了比较理想的表面离散深度图。     

Matching two perspective views

A computational approach to image matching is described. It uses multiple attributes associated with each image point to yield a generally overdetermined system of constraints, taking into account possible structural discontinuities and occlusions. In the algorithm implemented, intensity, edgeness, and cornerness attributes are used in conjunction with the constraints arising from intraregional smoothness, field continuity and discontinuity, and occlusions to compute dense displacement fields and occlusion maps along the pixel grids. The intensity, edgeness, and cornerness are invariant under rigid motion in the image plane. In order to cope with large disparities, a multiresolution multigrid structure is employed. Coarser level edgeness and cornerness measures are obtained by blurring the finer level measures. The algorithm has been tested on real-world scenes with depth discontinuities and occlusions. A special case of two-view matching is stereo matching, where the motion between two images is known. The algorithm can be easily specialized to perform stereo matching using the epipolar constraint     

基于摄像机纵向运动的序列图像的实时漫游

对于摄像机纵向运动的序列图像，提出了一种基于极线几何约束关系的当前视点目标图像生成算法：1）利用基于傅立叶变换的方法，得到远视点源图像中和近视点源图像无对应点的区域，并用一种背景色将其填充，得到填充后的远视点源图像；2）利用极线的整体匹配性质在两幅源图像中确定遍历整个源图像的对应源极线；3）将所有对应源极线按灰度进行分段；4）用动态规划匹配法确定对应源极线上段与段之间的匹配关系；5）通过两条对应源极线插值合成一条目标极线，生成当前视点的目标图像；6）对上一步得到的目标图像的背景色区域加以处理，得到最终的当前视点目标图像．提出了一种链表数据结构存储每相邻两幅源图像之间的预处理信息．漫游时实时读取．开发了一个图像漫游器，使用户能在其中实现不同视点的实时漫游．     

A Probabilistic Neural Network for Attribute Selection in Stereovision Matching

The key step in stereovision is image matching. This is carried out on the basis of selecting features, edge points, edge segments, regions, corners, etc. Once the features have been selected, a set of attributes (properties) for matching is chosen. This is a key issue in stereovision matching. This paper presents an approach for attribute selection in stereovision matching tasks based on a Probabilistic Neural Network, which allows the computation of a mean vector and a covariance matrix from which the relative importance of attributes for matching and the attribute interdependence can be derived. This is possible because the matching problem focuses on a pattern classification problem. The performance of the method is verified with a set of stereovision images and the results contrasted with a classical attribute selection method and also with the relevance concept. ID="A1" Correspondence and offprint requests to: Facultad de CC. Físicas, Universidad Complutense, 28040 Madrid, Spain. Email: pajares@dacya.ucm.es     

一种基于Hopfield网络的立体匹配方法

立体匹配是计算机视觉研究的经典难题，其算法的复杂度和精度直接影响了视觉系统对外部景物的重建性能。为此提出了一种新的基于神经网络的立体匹配方法，其基本思想是：在实现核线重排的前提下，利用唯一性、相容性以及相似性等匹配约束条件，建立反映对应极线间所有匹配点约束关系的能量函数，将其映射到二维Hopfield网络进行极小化求解，网络最后的稳态表示匹配点的对应关系；通过对图中所有极线进行上述操作，可以得到所求的视差图。与传统方法相比，本算法具有两个明显的特点：(1)匹配基元采用了普通的图像点，可以直接获得稠密的深度图；(2)Hopfield网的外部输入不再为常数，而是一个反映对应点灰度相似性关系的值。通过对合成图以及真实图景进行测试，验证了该方法的有效性。     

The non-parametric Parzen''s window in stereo vision matching

This paper presents an approach to the local stereovision matching problem using edge segments as features with four attributes. From these attributes we compute a matching probability between pairs of features of the stereo images. A correspondence is said true when such a probability is maximum. We introduce a nonparametric strategy based on Parzen's window (1962) to estimate a probability density function (PDF) which is used to obtain the matching probability. This is the main finding of the paper. A comparative analysis of other recent matching methods is included to show that this finding can be justified theoretically. A generalization of the proposed method is made in order to give guidelines about its use with the similarity constraint and also in different environments where other features and attributes are more suitable.     

宽基线图像特征点的立体匹配*

为了实现宽基线图像特征点的自动立体匹配,结合目前已有的算法,提出了一种新的分层匹配算法来获取最初的匹配点集,实现了基于对极几何约束的图像特征点自动提取及自动匹配。     

Contour matching using epipolar geometry

Matching features computed in images is an important process in multiview image analysis. When the motion between two images is large, the matching problem becomes very difficult. In this paper, we propose a contour matching algorithm based on geometric constraints. With the assumption that the contours are obtained from images taken from a moving camera with static scenes, we apply the epipolar constraint between two sets of contours and compute the corresponding points on the contours. From the initial epipolar constraints obtained from corner point matching, candidate contours are selected according to the epipolar geometry, contour end point constraints, and contour distance measures. In order to reduce the possibility of false matches, the number of match points on a contour is also used as a selection measure. The initial epipolar constraint is refined from the matched sets of contours. The algorithm can be applied to a pair or two pairs of images. All of the processes are fully automatic and successfully implemented and tested with various real images     

Epipolar geometry for prism-based single-lens stereovision

In order to simplify the design and implementation of a stereo vision system, prism has been used to capture stereo images with a single camera. This kind of system not only provides advantages over traditional two-camera stereo, but also reduces the complexity and cost of acquiring stereoscopic image. This paper investigated the characteristics of epipolar geometry for a single-lens prism-based stereovision. The prism was considered as a single optical lens. By analyzing each plane individually and then combining them together, an affine transformation matrix which can express the relationship between an object point and its image was derived. Then, the homography between object point and its image was established. Finally, the epipolar geometry as well as the epipolar rectification method was proposed. Experimental results verify that rectification of the image pair based on our proposed model can achieve better performance with much less geometric distortion.     

不变矩方法在区域匹配中的应用

提出了一种基于区域的立体匹配方法,该方法除了采用顺序性约束和惟一性约束外,根据外极线约束提出了外极带约束,采用比邻域约束更优越的相对位置约束,能有效地处理遮掩和区域变形情况下的双目立体匹配．在对多个候选匹配区域进行最佳匹配选择时,提出了重心距离和矩距离的概念,利用不变矩原理进行最佳候选区域选择．经实验证明,该方法是可行、有效的．     

Simultaneous two-view epipolar geometry estimation and motion segmentation by 4D tensor voting

We address the problem of simultaneous two-view epipolar geometry estimation and motion segmentation from nonstatic scenes. Given a set of noisy image pairs containing matches of n objects, we propose an unconventional, efficient, and robust method, 4D tensor voting, for estimating the unknown n epipolar geometries, and segmenting the static and motion matching pairs into n, independent motions. By considering the 4D isotropic and orthogonal joint image space, only two tensor voting passes are needed, and a very high noise to signal ratio (up to five) can be tolerated. Epipolar geometries corresponding to multiple, rigid motions are extracted in succession. Only two uncalibrated frames are needed, and no simplifying assumption (such as affine camera model or homographic model between images) other than the pin-hole camera model is made. Our novel approach consists of propagating a local geometric smoothness constraint in the 4D joint image space, followed by global consistency enforcement for extracting the fundamental matrices corresponding to independent motions. We have performed extensive experiments to compare our method with some representative algorithms to show that better performance on nonstatic scenes are achieved. Results on challenging data sets are presented.     

Improving stereovision matching through supervised learning

Most classical local stereovision matching algorithms use features representing objects in both images and compute the minimum difference attribute values. We have verified that the differences in attributes for the true matches cluster in a cloud around a centre. The correspondence is established on the basis of the minimum squared Mahalanobis distance between the difference of the attributes for a current pair of features and the cluster centre (similarity constraint). We introduce a new supervised learning strategy derived from the Learning Vector Quantization (LVQ) approach to get the best cluster centre. Additionally, we obtain the contribution or specific weight of each attribute for matching. We improve the learning law introducing a variable learning rate. The supervised learning and the improved learning law are the most important findings, which are justified by the computed better results compared with classical local stereovision matching methods without learning and with other learning strategies. The method is illustrated with 47 pairs of stereo images.     

一种基于角点检测的图像密集匹配算法

提出了一种鲁棒的图像自动立体匹配算法.利用Sobel算子对图像中的像素点进行检测,若是边缘点,则使用最小同值分割吸收核方法判断该点是否为角点.在两幅待匹配的图像间计算角点的梯度大小、梯度方向及灰度等的相似度,去除无法对应的角点,建立起待匹配图像中角点的对应关系,并计算基础矩阵.对基础矩阵进行迭代,去除误配点,计算出较精确的基础矩阵.由对极几何约束,采用动态规划方法,寻找左右两幅图像在对应极线上的所有像素点之间的对应,从而建立起两幅图像间像素点的密集匹配对应关系.试验结果表明,算法效果满意.     

Real-Time Feature Matching in Image Sequences for Non-Structured Environments. Applications to Vehicle Guidance

This paper describes an integrated vehicle control system with visual feedback. A general-purpose, low-level feature matching method, able to work in real time without any strict assumptions on the environment structure or camera parameters, generates low-level matching results, which are used as source of data for applications like mobile object tracking, among others. A generalized predictive path-tracking control approach keeps the vehicle on the trajectory defined by the moving target. In the low-level matching process, block-based features (windows) are selected and tracked along a stream of monocular images; least residual square error and similarity between clusters of features are used as constraints to select the right matching pair between multiple candidates. Real-time performance is achieved through optimized algorithms and a parallel DSP-based multiprocessor system implementation. Object detection and tracking is motion-based, and does not require a predefined model of the target. The integrated control system has been tested on the ROMEO-3R experimental vehicle.     

Using Stereo Matching with General Epipolar Geometry for 2D Face Recognition across Pose

Face recognition across pose is a problem of fundamental importance in computer vision. We propose to address this problem by using stereo matching to judge the similarity of two, 2D images of faces seen from different poses. Stereo matching allows for arbitrary, physically valid, continuous correspondences. We show that the stereo matching cost provides a very robust measure of similarity of faces that is insensitive to pose variations. To enable this, we show that, for conditions common in face recognition, the epipolar geometry of face images can be computed using either four or three feature points. We also provide a straightforward adaptation of a stereo matching algorithm to compute the similarity between faces. The proposed approach has been tested on the CMU PIE data set and demonstrates superior performance compared to existing methods in the presence of pose variation. It also shows robustness to lighting variation.