基于改进Census变换的多特性立体匹配算法

针对当前立体匹配算法存在的匹配准确率低，难以达到实用的高精度水平的问题，提出了一种基于改良的Census变换与色彩信息和梯度测度相结合的多特性立体匹配算法，实现高精度的双目立体匹配。算法首先在初始代价匹配阶段，将改进的Census变换、色彩和梯度测度赋权求和得出可靠的初始匹配代价；在聚合阶段，采取高效快捷的最小生成树聚合，获得匹配代价矩阵；最后根据胜者为王法则得到初始视差图，并引入左右一致性检测等策略优化视差图，获得高精度的视差图，实验阶段对源自Middlebury上的标准测试图进行测试验证，实验结果表明，经本文算法处理得到的15组测试数据集的视差图在非遮挡区域的平均误匹配率为6.81%，算法实时响应性优良。     

Accurate hardware-based stereo vision

To enable both accurate and fast real-time stereo vision in embedded systems, we propose a novel stereo matching algorithm that is designed for high efficiency when realized in hardware. We evaluate its accuracy using the Middlebury Stereo Evaluation, revealing its high performance at minimum tolerance. To outline the resource efficiency of the algorithm, we present its realization as an Intellectual Property (IP) core that is designed for the deployment in Field Programmable Gate Arrays (FPGAs) and Application Specific Integrated Circuits (ASICs).     

Real-time stereo matching based on fast belief propagation

In this paper, a global optimum stereo matching algorithm based on improved belief propagation is presented which is demonstrated to generate high quality results while maintaining real-time performance. These results are achieved using a foundation based on the hierarchical belief propagation architecture combined with a novel asymmetric occlusion handling model, as well as parallel graphical processing. Compared to the other real-time methods, the experimental results on Middlebury data show the efficiency of our approach.     

An improved graph cut algorithm in stereo matching

In stereo matching tasks, the matching effect is often very poor when the texture of the region is weak or repeated. To solve this problem, an improved Graph Cut stereo matching algorithm based on Census transform is proposed. The Hamming distance of the corresponding pixels in the left and right images after Census transform is introduced as the similarity measure in the data term of the energy function. In this way, the dependence on the pixel value is reduced. The stereo matching experiments are implemented on the standard images of Middlebury stereo benchmark and the real scene images, and it demonstrates that our algorithm is robust and can obtain better performance in weak texture or repeated texture region.     

结合传播滤波的立体匹配算法研究

针对局部立体匹配算法在边缘处容易出现误匹配的问题,本文提出了一种结合权值传播进行代价聚合的局部立体匹配方法。首先采用基于颜色梯度的绝对差及Census方法构造了匹配代价函数;然后,引入传播滤波平滑匹配代价的同时保持视差空间图像边缘,与其他局部滤波器相比,该滤波器利用可传播的权值思想,不受传统局部算法窗口大小的影响;最后,通过左右一致性检查和无效视差值填充获得最终视差图。实验表明,该方法在Middlebury Stereo数据集上可获得精确结果,与Middlebury测试平台上的IGF、TSGO和Dog-Guided算法相比平均误差最低。     

改进的跨尺度引导滤波立体匹配算法

作为双目三维重建中的关键步骤,双目立体匹配算法完成了从平面视觉到立体视觉的转化.但如何平衡双目立体匹配算法的运行速度和精度仍然是一个棘手的问题.本文针对现有的局部立体匹配算法在弱纹理、深度不连续等特定区域匹配精度低的问题,并同时考虑到算法实时性,提出了一种改进的跨多尺度引导滤波的立体匹配算法.首先融合AD和Census变换两种代价计算方法,然后采用基于跨尺度的引导滤波进行代价聚合,在进行视差计算时通过制定一个判断准则判断图像中每一个像素点的最小聚合代价对应的视差值是否可靠,当判断对应的视差值不可靠时,对像素点构建基于梯度相似性的自适应窗口,并基于自适应窗口修正该像素点对应的视差值.最后通过视差精化得到最终的视差图.在Middlebury测试平台上对标准立体图像对的实验结果表明,与传统基于引导滤波器的立体匹配算法相比具有更高的精度.     

A real-time global stereo-matching on FPGA

An improved global stereo matching algorithm is implemented on a single FPGA for real-time applications. Stereo matching is widely used in stereo vision systems, i.e. objects detection and autonomous vehicles. Global algorithms have much more accurate results than local algorithms, but global algorithms are not implemented on FPGA since they rely over high-end hardware resources. In this implementation the stereo pairs are divided into blocks, the hardware resources are reduced by processing one block once. The hardware implementation is based on a Xilinx Kintex 7 FPGA. Experiment results show that the proposed implementation has an accurate result for the Middlebury benchmarks and 30 frames per second (fps) @1920 × 1680 is achieved.     

A flexible and configurable GPGPU stereo matching framework

This paper presents a practical framework that consists of various local to global stereo matching algorithms on a general purpose computing on graphics processing units platform. The flexible framework provides users with a selection of individual sub-algorithms in each step of stereo matching. The framework runs on either a central processing unit or graphic processing unit across three different platforms, including a smartphone, embedded board, and desktop. On the basis of the proposed framework, we investigate different combinations of stereo matching algorithms for specific use cases. Accordingly, we provide the framework’s quantitative speed and accuracy analysis evaluated on the widely used stereo dataset. In addition, this paper also addresses the parallelization strategy on an embedded graphics processing unit. The experimental results show that the proposed framework is capable of real-time and accurate depth estimation of stereo input in video graphics array resolution on an embedded graphics processing unit.

     

Evaluation of stereo correspondence algorithms and their implementation on FPGA

The accuracy of stereo vision has been considerably improved in the last decade, but real-time stereo matching is still a challenge for embedded systems where the limited resources do not permit fast operation of sophisticated approaches. This work presents an evaluation of area-based algorithms used for calculating distance in stereoscopic vision systems, their hardware architectures for implementation on FPGA and the cost of their accuracies in terms of FPGA hardware resources. The results show the trade-off between the quality of such maps and the hardware resources which each solution demands, so they serve as a guide for implementing stereo correspondence algorithms in real-time processing systems.     

基于改进自适应权重的三边滤波立体匹配算法

针对基于双边滤波器（BF）的自适应权重（ASW）方法不能有效解决由视差不同但颜色相似的像素引起的模糊匹配问题,引入了一种新的基于三边滤波器（TF）的ASW方法,通过局部能量模型计算相邻像素之间的边界强度来提高匹配精度。为了提高匹配速度,将TF算法递归实现,把普通局部立体匹配算法的复杂度从[O(NWD)]降低为[O(N)]。在Middlebury基准测试集上进行实验并与其他局部立体匹配算法进行比较,RTF算法的平均误匹配率为4.91%,匹配精度高于同类型双目立体匹配算法,平均匹配速度达到258 ms,满足了双目立体匹配实时性的需求。     

Stereo correspondence using efficient hierarchical belief propagation

In this paper, a new algorithm is presented to compute the disparity map from a stereo pair of images by using Belief Propagation (BP). While many algorithms have been proposed in recent years, the real-time computation of an accurate disparity map is still a challenging task. The computation time and run-time memory requirements are two very important factors for all real-time applications. The proposed algorithm divides the matching process into two steps; they are initial matching and disparity map refinement. Initial matching is performed by memory efficient hierarchical belief propagation algorithm that uses less than half memory at run-time and minimizes the energy function at much faster rate as compare to other hierarchical BP algorithms that makes it more suitable for real-time applications. Disparity map refinement uses a simple but very effective single-pass approach that improves the accuracy without affecting the computation cost. Experiments by using Middlebury dataset demonstrate that the performance of our algorithm is the best among other real-time stereo matching algorithms.     

基于稀疏卷积的前景实时双目深度估计算法

为了提高立体匹配算法处理前景视差估计任务的计算效率,针对一般网络采用完全双目图像作为输入,场景内前景空间占比小而导致输入信息冗余度大的缺点,提出了一种基于稀疏卷积的目标实时立体匹配算法。为实现和改善算法对稀疏前景的视差估计,首先利用分割算法同时获得稀疏前景掩膜和场景语义特征;其次通过稀疏卷积提取稀疏前景区域的空间特征后与场景语义特征相融合,并将融合特征输入到解码模块进行视差回归;最后以前景真值图作为损失生成视差图。在ApolloScape数据集上的测试结果表明,所提算法的准确性和实时性均优于先进算法PSMNet和GANet,且算法的单次运行时间低至60.5 ms,对目标遮挡具有一定的鲁棒性,可用于目标实时深度估计。     

基于加权中值滤波的MST立体匹配算法

传统的最小生成树立体匹配算法对低纹理区域和遮挡区域不敏感,虽然最小生成树立体匹配算法后处理的中值滤波能够消除噪点,但是不能够消除边缘模糊。本文提出一种改进算法来克服这些局限性。首先,由于最小生成树匹配成本区分度不够高,研究并提出新最小生成树的匹配成本,使其可以减小不敏感区域的误匹配。其次,在后处理中使用加权中值滤波,以改善深度图像边缘。实验结果表明,在最小生成树立体匹配算法中使用改进匹配成本算法和加权中值滤波算法,在Middlebury数据集中平均误匹配率达到6.9%,本文算法在Middlebury和KITTI场景中都优于最小生成树立体匹配算法。     

一种改进的实时半全局立体匹配算法及硬件实现

在基于现场可编程门阵列的实时立体匹配系统中,Census变换算法针对特定区域的误匹配率较高。为提高匹配精度,提出一种具有高并行性流水线结构的实时半全局立体匹配算法并进行硬件实现。将改进的Tanimoto距离和带权重4方向的梯度绝对值差进行组合,作为新的初始匹配代价。在代价聚合阶段采用4路径并行结构的SGM算法,在视差选择阶段采用赢家通吃策略,在视差校正阶段采用阈值检测算法代替传统左右一致性检验算法。实验结果表明,该算法能够有效提高弱纹理和边缘区域的区分度,减少对中心点的依赖,降低资源占用,其在Middleburry平台上的平均误匹配率仅为7.52%,在Xilinx Zynq-7000平台上的匹配速率达到98 frame/s。     

Near-real-time stereo matching with slanted surface modeling and sub-pixel accuracy

This paper presents a new stereo matching algorithm which takes into consideration surface orientation at the per-pixel level. Two disparity calculation passes are used. The first pass assumes that surfaces in the scene are fronto-parallel and generates an initial disparity map, from which the disparity plane orientations of all pixels are estimated and refined. In the second pass, the matching costs for different pixels are aggregated along the estimated disparity plane orientations using adaptive support weights, where the support weights of neighboring pixels are calculated using a combination of four terms: a spatial proximity term, a color similarity term, a disparity similarity term, and an occlusion handling term. The disparity search space is quantized at sub-pixel level to improve the accuracy of the disparity results. The algorithm is designed for parallel execution on Graphics Processing Units (GPUs) for near-real-time processing speed. The evaluation using Middlebury benchmark shows that the presented approach outperforms existing real-time and near-real-time algorithms in terms of subpixel level accuracy.     

A progressive framework for dense stereo matching

A progressive framework is proposed for dense stereo matching to solve problems caused by weaktexture and occlusion in this paper. The main idea is that disparity is extracted progressively, from coarse to fine, from sparse to dense. First, a coarse disparity map is obtained by the segment-based pre-matching method, in which horizontal and vertical segment matching are performed in parallel and pre-matching results are merged to preserve more details. Second, disparity diffusion is performed to roughly estimate disparity values for miss-matched points. Third, a probabilistic approach is used for disparity refinement, taking into account stereo prior, image likehood and disparity smoothness. Experiments are made on the Middlebury benchmark to demostrate the effectiveness of the proposed algorithm.     

An automatized method to parameterize embedded stereo matching algorithms

Many applications rely on 3D information as a depth map. Stereo Matching algorithms reconstruct a depth map from a pair of stereoscopic images. Stereo Matching algorithms are computationally intensive, that is why implementing efficient stereo matching algorithms on embedded systems is very challenging for real-time applications.Indeed, like many vision algorithms, stereo matching algorithms have to set a lot of parameters and thresholds to work efficiently. When optimizing a stereo-matching algorithm, or changing algorithms parts, all those parameters have to be set manually. Finding the most efficient solution for a stereo-matching algorithm on a specific platform then becomes troublesome.This paper proposes an automatized method to find the optimal parameters of a dense stereo matching algorithm by learning from ground truth on a database in order to compare it with respect to any other alternative.Finally, for the C6678 platform, a map of the best compromise between quality and execution time is obtained, with execution times that are between 42 ms and 382 ms and output errors that are between 6% and 9.8%.     

FPGA based real-time on-road stereo vision system

Accurate and Real-Time stereo vision is an essential need for many computer vision applications, such as On-Road stereo vision system. In this paper, a fast and accurate system for On-Road stereo vision application is presented. In order to achieve this purpose, first, an algorithm is presented that is optimized for hardware implementation and On-Road application, then an appropriate hardware architecture for this algorithm is proposed. The approach uses gradient and Census transform as initial cost function, then cost is aggregate in cross-based supported reign. LR-check and disparity refinement is also utilized to improve final disparity. The proposed design is a standalone stereo vision system where all steps of algorithm is implemented on a hardware platform. Overall system is implemented on FPGA platform and it is tested on KITTI Database. The proposed system is also tested on Middlebury database without any changes in parameters. Experimental results show that the proposed system has high accuracy in KITTI and Middlebury database. In term of hardware resource, the system occupy %%66 of XC6VLX240T FPGA device for 1920 × 1080 image with 96 disparity levels that operated at 53.5 Fps.     

Massively parallel GPU computing for fast stereo correspondence algorithms

Current accurate stereo matching algorithms employ some key techniques that are not suitable for parallel GPU architecture. It will be tricky and cumbersome to directly take these techniques into GPU applications. Trying to tackle this difficulty, we design two GPU-based stereo matching algorithms, one using a local fixed aggregation window whose size is configurable, and the other using an adaptive aggregation window which only includes necessary pixels. We use the winner-takes-all (WTA) principle for optimization and a plain voting refinement for post-processing; both do not need complex data structures. We aim to implement on GPU platforms fast stereo matching algorithms that produce results with same-level quality as other WTA local dense methods that use window-based cost aggregation. In our GPU-based implementation of the fixed window partially demosaiced CFA stereo matching application, accelerations up to 20 times are obtained for large size images. In our GPU-based implementation of the adaptive window color stereo matching application, experiment results show that it can handle four pairs of standard images from Middlebury database within roughly 100 ms.