立体匹配算法进展

立体匹配是立体视觉中的重要环节;文章首先根据匹配算法使用的约束信息的不同把匹配算法分为局域算法和全局算法,并分别进行介绍,然后,针对目前各应用领域的要求,分别从准确性和实时性方面对立体匹配算法进行分析,从软件层面和硬件层面分析准确性和实时性要求的发展过程,最后,针对立体匹配算法中兼顾准确性和实时性要求的难题,结合作者的研究,指出了可能的解决途径和进一步研究的问题。     

基于区域匹配的实时加速技术

针对区域立体匹配计算量大实时性差的困难,分析了相关匹配算法的实际工作过程,采用消除冗余因子和Box滤波、多级分辨率匹配减小计算复杂度,对算法结构进行了改进和优化,并利用超线程和OpenMP技术对算法进行了加速,提出了一种实时区域匹配算法.对算法进行实验,结果表明算法符合了视觉导航的准确性和实时性要求,并且对于提高其他区域匹配算法实时性也具有重要借鉴意义.     

简化的SIFT算法在双目立体视觉中的应用

图像均匀匹配是双目立体视觉领域研究的重点。本文用SIFT特征匹配算法处理立体匹配,并利用构造圆环形窗口以及12维向量表示一个特征点的方法,既保持SIFT算法的尺度不变性,又有效降低了算法的复杂度,提高了算法实时性。     

一种应用于双目移动机器人目标跟踪的新型立体匹配算法

针对移动机器人目标跟踪对立体匹配准确性和实时性的要求,提出了一种基于平行配置系统的改进WTA算法;首先提取图像的边缘点和两幅视图间存在较大差异的点作为特征点;然后对特征点采用WTA算法进行立体匹配,而对非特征点仅进行简单的验证,其视差值为邻近像素的视差值;最后得到致密的视差图;该算法提取的特征点集中于视差不连续区域,实验结果表明该算法匹配精度与现有其它算法相当,但计算速度很好地满足了实时性的要求,并且边缘特性较好,是一种匹配准确、实时性好的立体匹配算法。     

一种基于图割理论的快速立体匹配算法

针对传统的立体匹配算法中存在的低纹理区域和遮挡区域匹配精度低、实时性不好等问题,提出了一种基于图割理论的立体匹配算法.把图像分割成色彩单一的不同区域;计算初始视差图,利用可靠点求取各分割区域的平面模板参数,对模板参数相同的相邻区域进行融合;构造全局能量函数,采用图割算法求取全局能量最小的视差最优分配.实验结果表明,该算法对低纹理区域和遮挡区域均有较好的匹配结果,能够满足高精度、高实时性的要求.     

立体匹配算法改进与优化

本文分析了当前主流的几类立体匹配算法,并通过对综合实时性与重构效果两方面内容的比较,得出半全局立体匹配算法的应用性更强。本文针对半全局匹配算法改进与优化,在代价计算部分将CIELAB色彩空间下的绝对灰度差与Census算法再结合梯度匹配计算方法作为代价计算函数,在代价聚合方面采用了引导滤波算法,并引入了多尺度聚合法,在不同尺度下分别进行代价计算,有效提高算法鲁棒性,使低纹理区域的块效应消除,图像边缘的匹配效果改善,误匹配率下降。     

基于改进自适应权重的三边滤波立体匹配算法

针对基于双边滤波器（BF）的自适应权重（ASW）方法不能有效解决由视差不同但颜色相似的像素引起的模糊匹配问题,引入了一种新的基于三边滤波器（TF）的ASW方法,通过局部能量模型计算相邻像素之间的边界强度来提高匹配精度。为了提高匹配速度,将TF算法递归实现,把普通局部立体匹配算法的复杂度从[O(NWD)]降低为[O(N)]。在Middlebury基准测试集上进行实验并与其他局部立体匹配算法进行比较,RTF算法的平均误匹配率为4.91%,匹配精度高于同类型双目立体匹配算法,平均匹配速度达到258 ms,满足了双目立体匹配实时性的需求。     

基于PatchMatch的半全局高效双目立体匹配算法

近年来双目立体匹配技术发展迅速,高精度、高分辨率、大视差的应用需求无疑对该技术的计算效率提出了更高的要求。由于传统立体匹配算法固有的计算复杂度正比于视差范围,已经难以满足高分辨率、大视差的应用场景。因此,从计算复杂度、匹配精度、匹配原理等多方面综合考虑,提出了一种基于PatchMatch的半全局双目立体匹配算法,在路径代价计算过程中使用空间传播机制,将可能的视差由整个视差范围降低为t个候选视差(t远远小于视差范围),显著减少了候选视差的数量,大幅提高了半全局算法的计算效率。对KITTI2015数据集的评估结果表明,该算法以5.81%的错误匹配率和20.2 s的匹配时间实现了准确性和实时性的明显提高。因此,作为传统立体匹配改进算法,该设计可以为大视差双目立体匹配系统提供高效的解决方案。     

基于TMS320C64x的立体匹配实时性问题研究

立体视觉是被动式测距的一种重要方法,而立体匹配则是整个体视算法中最重要也是最困难的部分.计算复杂度高,算法的实时性难以保证是立体匹配算法中的一个突出问题.针对基于高性能DSP的立体匹配实时性问题展开研究,提出了一种基于TMS320C64x的实时立体视觉算法.实验结果表明,提出的算法能够充分利用C64x DSP的架构特性,具有实时性高(大于每秒30帧),结果准确的优点.     

一种鲁棒的基于互信息的实时立体匹配算法

立体匹配算法依然是目前计算机视觉领域中的一个重要的研究热点.现有的绝大多数匹配算法都假定已满足相似性约束条件.然而在野外环境下,光照不均、场景为非朗伯表面、像机间差异都会使该约束条件无法满足,从而导致匹配失败.本文针对这种情况,并从实时应用的角度出发,提出一种鲁棒的基于互信息的实时立体匹配算法.该方法在相关匹配算法中引入互信息概念,并将其转换为相关算法可用的求和形式,通过迭代方法修正互信息值.然后,建立了三种亮度变化模型来验证该算法的有效性.实验结果表明,该算法能很好地抑制立体图像对间的亮度差异,具有很好的实时性与鲁棒性.     

An FPGA stereo matching unit based on fuzzy logic

Stereo matching is one of the most used algorithms in real-time image processing applications such as positioning systems for mobile robots, three-dimensional building mapping and recognition, detection and three-dimensional reconstruction of objects. In order to improve the performance, stereo matching algorithms often have been implemented in dedicated hardware such as FPGA or GPU devices. In this paper an FPGA stereo matching unit based on fuzzy logic is described. The proposed algorithm consists of three stages. First, three similarity parameters inherent to each pixel contained in the input stereo pair are computed. Then, the similarity parameters are sent to a fuzzy inference system which determines a fuzzy-similarity value. Finally, the disparity value is defined as the index which maximizes the fuzzy-similarity values (zero up to d_max). Dense disparity maps are computed at a rate of 76 frames per second for input stereo pairs of 1280 × 1024 pixel resolution and a maximum expected disparity equal to 15. The developed FPGA architecture provides reduction of the hardware resource demand compared to other FPGA-based stereo matching algorithms: near to 72.35% for logic units and near to 32.24% for bits of memory. In addition, the developed FPGA architecture increases the processing speed: near to 34.90% pixels per second and outperforms the accuracy of most of real-time stereo matching algorithms in the state of the art.     

Evaluation of stereo correspondence algorithms and their implementation on FPGA

The accuracy of stereo vision has been considerably improved in the last decade, but real-time stereo matching is still a challenge for embedded systems where the limited resources do not permit fast operation of sophisticated approaches. This work presents an evaluation of area-based algorithms used for calculating distance in stereoscopic vision systems, their hardware architectures for implementation on FPGA and the cost of their accuracies in terms of FPGA hardware resources. The results show the trade-off between the quality of such maps and the hardware resources which each solution demands, so they serve as a guide for implementing stereo correspondence algorithms in real-time processing systems.     

Stereo correspondence using efficient hierarchical belief propagation

In this paper, a new algorithm is presented to compute the disparity map from a stereo pair of images by using Belief Propagation (BP). While many algorithms have been proposed in recent years, the real-time computation of an accurate disparity map is still a challenging task. The computation time and run-time memory requirements are two very important factors for all real-time applications. The proposed algorithm divides the matching process into two steps; they are initial matching and disparity map refinement. Initial matching is performed by memory efficient hierarchical belief propagation algorithm that uses less than half memory at run-time and minimizes the energy function at much faster rate as compare to other hierarchical BP algorithms that makes it more suitable for real-time applications. Disparity map refinement uses a simple but very effective single-pass approach that improves the accuracy without affecting the computation cost. Experiments by using Middlebury dataset demonstrate that the performance of our algorithm is the best among other real-time stereo matching algorithms.     

A flexible and configurable GPGPU stereo matching framework

This paper presents a practical framework that consists of various local to global stereo matching algorithms on a general purpose computing on graphics processing units platform. The flexible framework provides users with a selection of individual sub-algorithms in each step of stereo matching. The framework runs on either a central processing unit or graphic processing unit across three different platforms, including a smartphone, embedded board, and desktop. On the basis of the proposed framework, we investigate different combinations of stereo matching algorithms for specific use cases. Accordingly, we provide the framework’s quantitative speed and accuracy analysis evaluated on the widely used stereo dataset. In addition, this paper also addresses the parallelization strategy on an embedded graphics processing unit. The experimental results show that the proposed framework is capable of real-time and accurate depth estimation of stereo input in video graphics array resolution on an embedded graphics processing unit.

     

Real-time stereo on GPGPU using progressive multi-resolution adaptive windows

We introduce a new GPGPU-based real-time dense stereo matching algorithm. The algorithm is based on a progressive multi-resolution pipeline which includes background modeling and dense matching with adaptive windows. For applications in which only moving objects are of interest, this approach effectively reduces the overall computation cost quite significantly, and preserves the high definition details. Running on an off-the-shelf commodity graphics card, our implementation achieves a 36 fps stereo matching on 1024 × 768 stereo video with a fine 256 pixel disparity range. This is effectively same as 7200 M disparity evaluations per second. For scenes where the static background assumption holds, our approach outperforms all published alternative algorithms in terms of the speed performance, by a large margin. We envision a number of potential applications such as real-time motion capture, as well as tracking, recognition and identification of moving objects in multi-camera networks.     

An automatized method to parameterize embedded stereo matching algorithms

Many applications rely on 3D information as a depth map. Stereo Matching algorithms reconstruct a depth map from a pair of stereoscopic images. Stereo Matching algorithms are computationally intensive, that is why implementing efficient stereo matching algorithms on embedded systems is very challenging for real-time applications.Indeed, like many vision algorithms, stereo matching algorithms have to set a lot of parameters and thresholds to work efficiently. When optimizing a stereo-matching algorithm, or changing algorithms parts, all those parameters have to be set manually. Finding the most efficient solution for a stereo-matching algorithm on a specific platform then becomes troublesome.This paper proposes an automatized method to find the optimal parameters of a dense stereo matching algorithm by learning from ground truth on a database in order to compare it with respect to any other alternative.Finally, for the C6678 platform, a map of the best compromise between quality and execution time is obtained, with execution times that are between 42 ms and 382 ms and output errors that are between 6% and 9.8%.     

基于FPGA的双目立体视觉系统

立体视觉的目的之一就是为了获得周围场景的3维信息,其关键在于匹配算法。然而即便是使用目前先进的通用处理器,其计算致密视差图所需的时间仍无法满足高速自主导航的需求。为了解决这个问题,提出了一种基于现场可编程门阵列(FPGA)的双目立体视觉系统的设计方案,同时介绍了系统的硬件结构,并在讨论区域匹配的快速算法的基础上,提出了基于FPGA的像素序列和并行窗口算法框架,用以实现零均值像素灰度差平方和(ZSSD)的匹配算法。该算法是先将视频信号经解码芯片生成场景立体图像对,并由FPGA来完成立体图像对的几何校正和ZSSD匹配算法,然后将获得的致密视差图通过PC I总线发送至上位机。实践表明,该算法效果好、速度快,不仅具有较强的鲁棒性,并且硬件系统性能稳定、可靠。此外,该方案还适用于像素灰度差的绝对值和(SAD)和像素灰度差的平方和(SSD)等多种传统区域匹配算法的快速实现和实时处理。     

行列双动态规划的改进自适应立体匹配算法

在各种立体匹配算法中,利用动态规划算法求解可有效地提高立体匹配的速度和精确度,同时具有实时性好、易于实现的优点。利用动态规划算法的优点,提出一种基于行列动态规划的自适应立体匹配算法,采用改进的自适应代价函数和能量最小化模型,对最优化问题进行求解。在求解的过程中,基于行动态规划得到的列方向视差值的变化给予对应数据项不同的奖励值,以减少行动态规划产生的明显条纹,最后使用列动态规划得出最终结果。实验结果表明,该算法能够减少总体的匹配错误率,减少明显的条纹瑕疵,取得较理想的立体匹配效果。     

图像立体匹配研究进展

图像的立体匹配一直是立体视觉的研究重点.首先简要介绍了立体匹配方法及其分类,归纳了立体匹配中的各种约束条件和相似性测度函数;然后总结了局部匹配算法和全局匹配算法的特点,并结合对象的三维重建问题重点分析了全局匹配算法中的动态规划算法、图割法和置信度传播算法;最后对立体匹配研究面临的主要问题给出了一些建议.     

A fast stereo matching algorithm suitable for embedded real-time systems

In this paper, the challenge of fast stereo matching for embedded systems is tackled. Limited resources, e.g. memory and processing power, and most importantly real-time capability on embedded systems for robotic applications, do not permit the use of most sophisticated stereo matching approaches. The strengths and weaknesses of different matching approaches have been analyzed and a well-suited solution has been found in a Census-based stereo matching algorithm. The novelty of the algorithm used is the explicit adaption and optimization of the well-known Census transform in respect to embedded real-time systems in software. The most important change in comparison with the classic Census transform is the usage of a sparse Census mask which halves the processing time with nearly unchanged matching quality. This is due the fact that large sparse Census masks perform better than small dense masks with the same processing effort. The evidence of this assumption is given by the results of experiments with different mask sizes. Another contribution of this work is the presentation of a complete stereo matching system with its correlation-based core algorithm, the detailed analysis and evaluation of the results, and the optimized high speed realization on different embedded and PC platforms. The algorithm handles difficult areas for stereo matching, such as areas with low texture, very well in comparison to state-of-the-art real-time methods. It can successfully eliminate false positives to provide reliable 3D data. The system is robust, easy to parameterize and offers high flexibility. It also achieves high performance on several, including resource-limited, systems without losing the good quality of stereo matching. A detailed performance analysis of the algorithm is given for optimized reference implementations on various commercial of the shelf (COTS) platforms, e.g. a PC, a DSP and a GPU, reaching a frame rate of up to 75 fps for 640 × 480 images and 50 disparities. The matching quality and processing time is compared to other algorithms on the Middlebury stereo evaluation website reaching a middle quality and top performance rank. Additional evaluation is done by comparing the results with a very fast and well-known sum of absolute differences algorithm using several Middlebury datasets and real-world scenarios.