GOLD: a parallel real-time stereo vision system for genericobstacle and lane detection

This paper describes the generic obstacle and lane detection system (GOLD), a stereo vision-based hardware and software architecture to be used on moving vehicles to increment road safety. Based on a full-custom massively parallel hardware, it allows to detect both generic obstacles (without constraints on symmetry or shape) and the lane position in a structured environment (with painted lane markings) at a rate of 10 Hz. Thanks to a geometrical transform supported by a specific hardware module, the perspective effect is removed from both left and right stereo images; the left is used to detect lane markings with a series of morphological filters, while both remapped stereo images are used for the detection of free-space in front of the vehicle. The output of the processing is displayed on both an on-board monitor and a control-panel to give visual feedbacks to the driver. The system was tested on the mobile laboratory (MOB-LAB) experimental land vehicle, which was driven for more than 3000 km along extra-urban roads and freeways at speeds up to 80 km/h, and demonstrated its robustness with respect to shadows and changing illumination conditions, different road textures, and vehicle movement.     

基于双目立体视觉测量原理的物体运动速度测量

采用双目立体视觉测量系统进行炮弹速度的测量, 实现一种利用无速度测量因素测量设备测速的新方法。选择合适参数的两 台摄像机获得被测物体清晰的像,将两台摄像机搭建成光轴平行的双目视觉测量 系统,获得炮弹在运行过程中的每一时刻的三维轮廓数据,根据获得的三维数据进 行空间位置节点的三维插值,得到炮弹在t时间段内运行的距离,并求取平均 速度。仿真实验结果表明,本文方法可以准确、快速利用无速度测量因素的测量 设备获得炮弹的运行速度。     

ETS-3D: An Efficient Two-Stage Framework for Stereo 3D Object Detection

We propose an efficient two-stage framework for stereo 3D object detection, called ETS-3D. Contrary to many recent approaches that rely on depth maps predicted using time-consuming stereo matching models, our approach utilizes the well-designed features to generate high-quality 3D proposals in stage-1, without explicitly exploiting predicted depth map. Specifically, we leverage pixel-wise correlation to produce normalized cost volumes to weight the left image features, and fuse multi-scale weighted features to obtain the weighted and fused features for 3D proposal generation. To maintain fast computation, only the filtered positive 3D proposals are fed into the stage-2 sub-network for further proposal refinement and quality prediction. Furthermore, we reconstruct the 3D proposal features in stage-2 to make use of different feature representations, achieving more accurate detection results. The experimental results on the KITTI 3D object detection benchmark demonstrate that our method achieves state-of-the-art performance, and can run at more than 10 fps.     

车型识别系统中利用阴影特点快速定位车体

阴影在车型识别系统中对于车体的准确定位产生很大的影响，为了能准确地定位车体，提出一种快速定位阴影与车体之间边界的算法即根据平行投影知识和原理的方法不定位阴影与车体之间的边界，同时利用该算法也能找出货车本身所具有的特征和华车的车头和车尾的定位，利用所寻找的特征点也可以进一步判断车所属的大致类型。利用该算法进行了大量的实验，从实验结果来看，该算法具有较好的稳定性和有效性。从该算法的计算时间来看也满足实时系统的需要。     

Vision-based vehicles in Japan: machine vision systems and drivingcontrol systems

This paper surveys three intelligent vehicles developed in Japan, and in particular the configurations, the machine vision systems, and the driving control systems. The first one is the Intelligent Vehicle, developed since the mid 1970's, which has a machine vision system for obstacle detection and a dead reckoning system for autonomous navigation on a compact car. The machine vision system with stereo TV cameras is featured by real time processing using hard-wired logic. The dead reckoning function and a new lateral control algorithm enable the vehicle to drive from a starting point to a goal. It drove autonomously at about 10 km/h while avoiding an obstacle. The second one is the Personal Vehicle System (PVS), developed in the late 1980's, which is a comprehensive test system for a vision-based vehicle. The machine vision system captures lane markings at both road edges along which the vehicle is guided. The PVS has another machine vision system for obstacle detection with stereo cameras. The PVS drove at 10-30 km/h along lanes with turnings and crossings. The third one is the Automated Highway Vehicle System (AHVS) with a single TV camera for lane-keeping by PD control. The machine vision system uses an edge extraction algorithm to detect lane markings. The AHVS drove at 50 km/h along a lane with a large curvature     

基于车底阴影的车前障碍物检测

基于计算机视觉的道路车辆检测是智能车辆导航的核心问题,实时准确地检测前方运动车辆的位置信息是车辆安全驾驶的前提。文中采用车底阴影的前方运动车辆检测算法,在基于车道线检测的基础上,通过车底阴影检测,实时准确地检测前方车辆。该算法通过使用Otsu阈值分割提取出车道线,生成AOI区域,再进行两次自适应阈值分割提取车底与路面的交线,从而检测出前方运动车辆。经过在高速公路上对运动车辆检测实验证明,该算法基本满足车辆安全驾驶的需求,并能准确实时地检测出前方运动车辆,进而减少了交通事故的发生。     

多环境下的实时前车检测与车距测量

本文利用智能手机对视频进行实时处理,从而实现对前车的检测和车距的估算。采用两步前车检测技术,即寻找车辆假设区域和验证车辆假设区域。通过基于车道先验的两步阈值法提取车底阴影,通过车道先验对阴影进行筛选从而寻找车辆假设区域,并利用基于Haar-like特征的Adaboost方法来对车辆假设区域进行验证,之后通过视频流特征对误检进行剔除并对车辆进行跟踪。本文还结合摄像头透视几何关系,只需用户提供摄像头高度信息即可对前车距离进行准确估算,可以作为有效的危险预警手段。      

基于光栅投射的机器人导航视觉传感器研究

为了实现暗环境下移动机器人导航中障碍物的检测与运动机器人的定位,采用了一种组合式光栅投射立体视觉传感器研究方法,首先通过光栅投射和立体视觉相融合的方法,建立光栅投射立体视觉传感器几何和数学模型,然后利用空间设备位置约束原理和投影平面相交的方法,进行了机器人视场内空间物体的3维坐标计算,建立了可靠真实的障碍物检测和分析方法,并进行了理论分析和实验验证,取得了距离计算精度0.8mm的数据。结果表明,该方法对于图像计算的精度在亚像素级。该方法有利于目前黑暗环境中机器人无法自主导航难题的突破,为黑暗环境中无全球定位系统支持的机器人导航提供了基础探索。     

基于散斑立体匹配的快速三维人脸重建

本文提出一种高效的人脸三维重建方法。该方法将 散斑投影至人脸表面以增加其特征信息,并采用一种由粗到精的时空立体匹配算法来提高三 维人脸重建的精度。进一步,该算法利用 时空积分图对立体匹配的代价函数进行加速计算,进而提高了重建效率。此外,所提出方法 通过人脸检测 去除无关背景,使后续的三维重建算法能够高效地作用在人脸区域上。实验表明,所提出方 法对3D打印人 脸(精度为0.01 mm)模型的重建平均误差为0.32 mm,对哑铃规球心测距和直径测距(精度皆为0.01 mm)其 误差皆低于1个百分点,以上结果优于同类产品。与现有立体匹配算法相比,本文方法所得 视差图面部无 空洞且视差变化均匀,更真实地反映出被测人脸的三维形状。     

Automatic Moving Vehicle Detection and Classification Based on Artificial Neural Fuzzy Inference System

The main objective of this work is to automatically detect moving vehicles on the road. Adaptive Neuro-Fuzzy Inference System (ANFIS) classifier is adopted in this paper to classify moving vehicles on road. An input traffic video scenes are taken with vertical and horizontal positioned cameras. The proposed system contains six major steps such as preprocessing, vehicle detection, tracking, structural matching, feature extraction and classification. In this proposed method, preprocessing consists of color conversion and noise removal. Vehicle detection is performed by using background subtraction and Otsu thresholding algorithm. Kalman filter is used in the third step to track moving vehicles in successive frames. In the fourth step, Active Shape Modelling method is used to recover the 3D shape of the vehicle in order to find the boundaries of vehicle. In the fifth step, features of the detected vehicles are extracted by Harrish corner detector, log Gabor filter and these features are taken into account to classify the types of vehicle. Finally, ANFIS is proposed to classify the vehicles which is trained by updating the membership function. Experimentation results provides better accuracy rate and low mean error rate when compared with the state of art methods.     

融合梯度特性与置信度的立体匹配算法

针对现有局部立体匹配算法在计算匹配代价时, 不能很好区分强弱纹理区域,及在视差计算过程 中,不能很好的解决视差歧义问题,提出一种融合梯度特性与置信度的立体匹配算法。首先 计算梯度特 征,并根据梯度特征信息选择匹配代价计算的匹配窗口,针对强弱不同纹理区域选择不同尺 寸的匹配窗 口,有效的提高了立体匹配精度,降低了误匹配率;然后在视差计算中引入置信度约束条件 ,解决了视差 计算中视差歧义的问题,提高了立体匹配算法的稳定性与精度;最后使用水平与垂直方向交 叉区域检测进 行奇异值的修正。实验结果表明,该算法在Middlebury数据集中31对 立体图像对的平均误匹配率为7.96%,有效的提高了立体匹配精度。     

LIDAR Sensing for Vehicle Lateral Guidance: Algorithm and Experimental Study

This paper describes a new vehicle detection system based on a laser imaging, detection, and ranging (LIDAR) sensor, and conducts an experimental study to investigate the sensor's role in the current California PATH vehicle lateral guidance systems. The LIDAR sensor is installed on a controlled vehicle and it can measure the relative distance of the vehicle from a preceding vehicle, by scanning the horizontal plane with laser beams. Environmental clutter becomes the main challenge in data processing, when LIDAR tries to track the desired target. A probabilistic data association-based algorithm has been developed to solve this problem, which has been verified in real-time experiments using two Buick LaSabre vehicles. The experimental study also reveals the relation between the LIDAR outputs and the magnetic reference system widely used by the current PATH lateral control systems, and the results provide the guidelines on how this new sensor system may be used for vehicle lateral guidance     

Human Detection for Video Surveillance in Hospital

This paper presents a human detection system in a vision-based hospital surveillance environment. The system is composed of three subsystems, i.e. background segmentation subsystem (BSS), human feature extraction subsystem (HFES), and human recognition subsystem (HRS). The codebook background model is applied in the BSS, the histogram of oriented gradients (HOG) features are used in the HFES, and the support vector machine (SVM) classification is employed in the HRS. By means of the integration of these subsystems, the human detection in a vision-based hospital surveillance environment is performed. Experimental results show that the proposed system can effectively detect most of the people in hospital surveillance video sequences.     

基于增强型轻量级网络的车载热成像目标检测方法

车载热成像系统不依赖光源,对天气状况不敏感,探测距离远,对夜间行车有很大辅助作用,热成像自动目标检测对夜间智能驾驶具有重要意义。车载热成像系统所采集的红外图像相比可见光图像具有分辨率低,远距离小目标细节模糊的特点,且热成像目标检测方法需考虑车辆移动速度所要求的算法实时性以及车载嵌入式平台的计算能力。针对以上问题,本文提出了一种针对热成像系统的增强型轻量级红外目标检测网络（Infrared YOLO,I-YOLO）,该网络采用（Tiny you only look once,Tiny-YOLO V3）的基础结构,根据红外图像特点,提取浅层卷积层特征,提高红外小目标检测能力,使用单通道卷积核,降低运算量,检测部分使用基于CenterNet结构的检测方式以降低误检测率,提高检测速度。经实际测试,Enhanced Tiny-YOLO目标检测网络在热成像目标检测方面,平均检测率可达91%,检测平均速度达到81Fps,训练模型权重96MB,适宜于车载嵌入式系统上部署。     

Vehicle detection using normalized color and edge map.

This paper presents a novel vehicle detection approach for detecting vehicles from static images using color and edges. Different from traditional methods, which use motion features to detect vehicles, this method introduces a new color transform model to find important "vehicle color" for quickly locating possible vehicle candidates. Since vehicles have various colors under different weather and lighting conditions, seldom works were proposed for the detection of vehicles using colors. The proposed new color transform model has excellent capabilities to identify vehicle pixels from background, even though the pixels are lighted under varying illuminations. After finding possible vehicle candidates, three important features, including corners, edge maps, and coefficients of wavelet transforms, are used for constructing a cascade multichannel classifier. According to this classifier, an effective scanning can be performed to verify all possible candidates quickly. The scanning process can be quickly achieved because most background pixels are eliminated in advance by the color feature. Experimental results show that the integration of global color features and local edge features is powerful in the detection of vehicles. The average accuracy rate of vehicle detection is 94.9%.     

Stereo image analysis for multi-viewpoint telepresence applications

An improved method for combined motion and disparity estimation in stereo sequences to synthesize temporally and perspectively intermediate views is presented. The main problems of matching methods for motion and disparity analysis are summarised. The improved concept is based on a modified block matching algorithm in which a cost function consisting of feature- and area-based correlation together with an appropriately weighted temporal smoothness term is applied. Considerable improvements have been obtained with respect to the motion and disparity assignments by introducing a confidence measure to evaluate the reliability of estimated correspondences. In occluded image areas, enhanced results are obtained applying an edge-assisted vector interpolation strategy. Two different image synthesis concepts are presented. The first concept is suitable for processing natural stereo sequences. It comprises the detection of covered and uncovered image areas caused by motion or disparity. This information is used to switch between different interpolation and extrapolation modes during the computation of intermediate views. The proposed object-based approach is suitable for processing typical video conference scenes containing extremely large occluded image regions and keeping implementation costs low. A set of stereo sequences has been processed. The performed computer simulations show that a continuous motion parallax can be obtained with good image quality by using sequences taken with stereo cameras having large interaxial distances.     

3-D-Skeleton-Based Head Detection and Tracking Using Range Images

Vision-based 3-D head detection and tracking systems have been studied in several applications like video surveillance, face-detection systems, and occupant posture analysis. In this paper, we present the development of a topology-based framework using a 3-D skeletal model for the robust detection and tracking of a vehicle occupant's head position from low-resolution range image data for a passive safety system. Unlike previous approaches to head detection, the proposed approach explores the topology information of a scene to detect the position of the head. Among the different available topology representations, the Reeb graph technique is chosen and is adapted to low-resolution 3-D range images. Invariance of the graph under rotations is achieved by using a Morse radial distance function. To cope with the particular challenges such as the noise and the large variations in the density of the data, a voxel neighborhood connectivity notion is proposed. A multiple-hypothesis tracker (MHT) with nearest-neighbor data association and Kalman filter prediction is applied on the endpoints of the Reeb graph to select and filter the correct head candidate out of Reeb graph endpoints. A systematic evaluation of the head detection framework is carried out on full-scale experimental 3-D range images and compared with the ground truth. It is shown that the Reeb graph topology algorithm developed herein allows the correct detection of the head of the occupant with only two head candidates as input to the MHT. Results of the experiments demonstrate that the proposed framework is robust under the large variations of the scene. The processing requirements of the proposed approach are discussed. It is shown that the number of operations is rather low and that real-time processing requirements can be met with the proposed method.     

A least-squares-based 2-D filtering scheme for stereo imagecompression

A two-dimensional (2-D) least squares (LS)-based filtering scheme for high fidelity stereo image compression applications is introduced in this correspondence. This method removes the effects of mismatching in a stereo image pair by applying the left image as the reference input to a 2-D transversal filter while the right image is used as the desired output. The weights of the filter are computed using a block-based LS method. A reduced order filtering scheme is also introduced to find the optimum number of filter coefficients. The principal coefficients and the disparity vectors are used together with left image to reconstruct the right image at the receiver, The proposed schemes are examined on a real stereo image pair for 3D-TV applications and the results were benchmarked against those of the block-matching method.     

Rigid-motion-invariant classification of 3-D textures

This paper studies the problem of 3-D rigid-motion-invariant texture discrimination for discrete 3-D textures that are spatially homogeneous by modeling them as stationary Gaussian random fields. The latter property and our formulation of a 3-D rigid motion of a texture reduce the problem to the study of 3-D rotations of discrete textures. We formally develop the concept of 3-D texture rotations in the 3-D digital domain. We use this novel concept to define a "distance" between 3-D textures that remains invariant under all 3-D rigid motions of the texture. This concept of "distance" can be used for a monoscale or a multiscale 3-D rigid-motion-invariant testing of the statistical similarity of the 3-D textures. To compute the "distance" between any two rotations R(1) and R(2) of two given 3-D textures, we use the Kullback-Leibler divergence between 3-D Gaussian Markov random fields fitted to the rotated texture data. Then, the 3-D rigid-motion-invariant texture distance is the integral average, with respect to the Haar measure of the group SO(3), of all of these divergences when rotations R(1) and R(2) vary throughout SO(3). We also present an algorithm enabling the computation of the proposed 3-D rigid-motion-invariant texture distance as well as rules for 3-D rigid-motion-invariant texture discrimination/classification and experimental results demonstrating the capabilities of the proposed 3-D rigid-motion texture discrimination rules when applied in a multiscale setting, even on very general 3-D texture models.     

Fast block matching algorithm based on the winner-update strategy

Block matching is a widely used method for stereo vision, visual tracking, and video compression. Many fast algorithms for block matching have been proposed in the past, but most of them do not guarantee that the match found is the globally optimal match in a search range. This paper presents a new fast algorithm based on the winner-update strategy which utilizes an ascending lower bound list of the matching error to determine the temporary winner. Two lower bound lists derived by using partial distance and by using Minkowski's inequality are described. The basic idea of the winner-update strategy is to avoid, at each search position, the costly computation of the matching error when there exists a lower bound larger than the global minimum matching error. The proposed algorithm can significantly speed up the computation of the block matching because (1) computational cost of the lower bound we use is less than that of the matching error itself; (2) an element in the ascending lower bound list will be calculated only when its preceding element has already been smaller than the minimum matching error computed so far; (3) for many search positions, only the first several lower bounds in the list need to be calculated. Our experiments have shown that, when applying to motion vector estimation for several widely-used test videos, 92% to 98% of operations can be saved while still guaranteeing the global optimality. Moreover, the proposed algorithm can be easily modified either to meet the limited time requirement or to provide an ordered list of best candidate matches.