Faster CNN-based vehicle detection and counting strategy for fixed camera scenes

Automatic detection and counting of vehicles in a video is a challenging task and has become a key application area of traffic monitoring and management. In this paper, an efficient real-time approach for the detection and counting of moving vehicles is presented based on YOLOv2 and features point motion analysis. The work is based on synchronous vehicle features detection and tracking to achieve accurate counting results. The proposed strategy works in two phases; the first one is vehicle detection and the second is the counting of moving vehicles. Different convolutional neural networks including pixel by pixel classification networks and regression networks are investigated to improve the detection and counting decisions. For initial object detection, we have utilized state-of-the-art faster deep learning object detection algorithm YOLOv2 before refining them using K-means clustering and KLT tracker. Then an efficient approach is introduced using temporal information of the detection and tracking feature points between the framesets to assign each vehicle label with their corresponding trajectories and truly counted it. Experimental results on twelve challenging videos have shown that the proposed scheme generally outperforms state-of-the-art strategies. Moreover, the proposed approach using YOLOv2 increases the average time performance for the twelve tested sequences by 93.4% and 98.9% from 1.24 frames per second achieved using Faster Region-based Convolutional Neural Network (F R-CNN ) and 0.19 frames per second achieved using the background subtraction based CNN approach (BS-CNN ), respectively to 18.7 frames per second.

     

融合特征的卫星视频车辆单目标跟踪

目的 卫星视频作为新兴遥感数据,可以提供观测区域高分辨率的空间细节信息与丰富的时序变化信息,为交通监测与特定车辆目标跟踪等应用提供了不同于传统视频视角的信息。相较于传统视频数据,卫星视频中的车辆目标分辨率低、尺度小、包含的信息有限。因此,当目标边界不明、存在部分遮挡或者周边环境表观模糊时,现有的目标跟踪器往往存在严重的目标丢失问题。对此,本文提出一种基于特征融合的卫星视频车辆核相关跟踪方法。方法 对车辆目标使用原始像素和方向梯度直方图（histogram of oriented gradient,HOG）方法提取包含互补判别能力的特征,利用核相关目标跟踪器分别得到具备不变性和判别性的响应图;通过响应图融合的方式结合两种特征的互补信息,得到目标位置;使用响应分布指标（response distribution criterion,RDC）判断当前目标特征的稳定性,决定是否更新跟踪器的表征模型。本文使用的相关滤波方法具有计算量小且运算速度快的特点,具备跟踪多个车辆目标的拓展能力。结果 在8个卫星视频序列上与主流的6种相关滤波跟踪器进行比较,实验数据涵盖光照变化、快速转弯、部分遮挡、阴影干扰、道路颜色变化和相似目标临近等情况,使用准确率曲线和成功率曲线的曲线下面积（area under curve,AUC）对车辆跟踪的精度进行评价。结果表明,本文方法较好地均衡了使用不同特征的基础跟踪器（性能排名第2）的判别能力,准确率曲线AUC提高了2.9%,成功率曲线AUC下降了4.1%,成功跟踪车辆目标,不发生丢失,证明了本文方法的先进性和有效性。结论 本文提出的特征融合的卫星视频车辆核相关跟踪方法,均衡了不同特征提取器的互补信息,较好解决了卫星视频中车辆目标信息不足导致的目标丢失问题,提升了精度。     

一种基于灰度直方图的交通检测系统

在中国由于行人和机动车混行的情况非常普遍,因此行人的检测和机动车一样也是交通检测的重要方面。然而国内外在行人与机动车混合交通流检测方面的研究却不多见。文章提出并阐述了一套基于视频的综合交通检测系统。该系统通过视频技术实时检测获得路段或路口上机动车、行人的综合交通数据。系统运用了实用性很强自适应背景差减法进行前景图像分割,并提出了基于灰度直方图的目标跟踪方法,在一定程度上解决了在人车混行中比较严重的遮挡问题。实验结果表明,系统在前景图像提取以及目标跟踪分类等环节都取得了令人满意的成果。     

基于多属性层次识别的车辆视频 检索系统设计研究

交通监控视频中车辆的有效检测和实时跟踪,是车辆行为分析和识别的前提,也是智能交通系统（ITS） 的核心内容和关键技术。本文在深入分析当前车辆属性识别方法以及车辆视频检索关键技术的基础上,结合交通监控 视频的自身特点,从应用的角度出发,设计一款融合车牌、车身颜色、车型等多个车辆外观属性进行层次识别的机动车辆 检索系统。该系统可为用户提供多种查询方式,从而实现交通监控视频中相关机动车辆的准确检索。     

A Multimedia Data Mining Framework: Mining Information from Traffic Video Sequences

The analysis and mining of traffic video sequences to discover important but previously unknown knowledge such as vehicle identification, traffic flow, queue detection, incident detection, and the spatio-temporal relations of the vehicles at intersections, provide an economic approach for daily traffic monitoring operations. To meet such demands, a multimedia data mining framework is proposed in this paper. The proposed multimedia data mining framework analyzes the traffic video sequences using background subtraction, image/video segmentation, vehicle tracking, and modeling with the multimedia augmented transition network (MATN) model and multimedia input strings, in the domain of traffic monitoring over traffic intersections. The spatio-temporal relationships of the vehicle objects in each frame are discovered and accurately captured and modeled. Such an additional level of sophistication enabled by the proposed multimedia data mining framework in terms of spatio-temporal tracking generates a capability for automation. This capability alone can significantly influence and enhance current data processing and implementation strategies for several problems vis-à-vis traffic operations. Three real-life traffic video sequences obtained from different sources and with different weather conditions are used to illustrate the effectiveness and robustness of the proposed multimedia data mining framework by demonstrating how the proposed framework can be applied to traffic applications to answer the spatio-temporal queries.     

基于多特征融合的视频交通数据采集方法

提出了一种基于多特征融合的视频交通数据采集方法, 核心思想是: 在图像中设置虚拟线圈, 假设车辆从虚拟线圈上驶过时引起像素变化, 通过识别这种像素变化来检测车辆并估计车速. 与现有技术相比, 本文的贡献在于: 1) 综合利用虚拟线圈内的前景面积、纹理变化、像素运动等特征来检测车辆, 提出了有效的多特征融合方法, 显著提高了车辆检测精度; 2) 根据单个虚拟线圈内的像素运动向量来估计车速, 避免了双线圈测速法的错误匹配问题. 算法测试结果表明本文算法能够在复杂多样的交通场景和天气条件下, 准确地检测车辆和估计车速. 在算法研究的基础上, 研制了一款嵌入式交通视频检测器, 在路口长期采集交通数据, 为交通信号控制和交通规律分析提供决策依据.     

基于稀疏帧检测的交通目标跟踪

为了获取高速公路交通视频中目标车辆的行驶轨迹,提出一种基于视频的多目标车辆跟踪及实时轨迹分布算法,为交通管理系统和交通决策提供目标车辆交通信息.首先,使用YOLOv4算法检测目标车辆位置及置信度.其次,在不同场景条件下,使用提出的基于稀疏帧检测的跟踪方法,结合KCF跟踪算法,将车辆数据进行关联获取完整轨迹.最后,用车辆分布图和交通场景俯视图显示轨迹,便于交通管理与分析.实验结果表明,提出的跟踪方法在车辆跟踪中有较高的跟踪正确率,同时基于稀疏帧检测的跟踪方法处理速度也较快,实时轨迹分布正确反映了真实场景的车道信息以及目标车辆运动信息.     

Multiframe Many–Many Point Correspondence for Vehicle Tracking in High Density Wide Area Aerial Videos

This paper presents a novel framework for tracking thousands of vehicles in high resolution, low frame rate, multiple camera aerial videos. The proposed algorithm avoids the pitfalls of global minimization of data association costs and instead maintains multiple object-centric associations for each track. Representation of object state in terms of many to many data associations per track is proposed and multiple novel constraints are introduced to make the association problem tractable while allowing sharing of detections among tracks. Weighted hypothetical measurements are introduced to better handle occlusions, mis-detections and split or merged detections. A two-frame differencing method is presented which performs simultaneous moving object detection in both. Two novel contextual constraints of vehicle following model, and discouragement of track intersection and merging are also proposed. Extensive experiments on challenging, ground truthed data sets are performed to show the feasibility and superiority of the proposed approach. Results of quantitative comparison with existing approaches are presented, and the efficacy of newly introduced constraints is experimentally established. The proposed algorithm performs better and faster than global, 1–1 data association methods.     

矩阵低秩表示的目标跟踪算法

目的 目标跟踪中,遮挡、强烈光照及运动模糊等干扰对跟踪精度的影响较大,其为目标外观的观测建模精度带来一定的困难。此外,很多现有算法在观测建模中都以向量形式表示样本数据,使得样本数据原有结构及其各像素的潜在关系被有意改变,从而导致观测模型数据维度及计算复杂度的提高。方法 本文通过深入研究跟踪框架的观测建模问题,提出一种新颖的基于矩阵低秩表示的观测建模方法及其相应的似然度测度函数,使得跟踪算法能够充分挖掘样本数据的潜在特征结构,从而更加精确探测目标在遮挡或强烈光照等各种复杂干扰下的外观变化。同时,以矩阵形式表述样本信号的数据格式,使得其视觉特征的空间分布保留完好,并有效降低数据维度和计算复杂度。结果 本文跟踪算法在富有挑战性干扰因素的跟踪环境中体现出更为鲁棒的跟踪性能,能够较好地解决跟踪中遮挡或强烈光照所引起的模型退化和漂移等问题。在10个经典测试视频中,本文跟踪算法的平均中心点误差为5.29像素,平均跟踪重叠率为78%,平均跟踪成功率为98.28%,均优于其他同类算法。结论 本文以2维矩阵数据原型为载体,提出了一种新的多任务观测建模框架和最大似然度估计模型。实验数据的定性与定量分析结果表明,本文算法与一些优秀的同类算法相比,其跟踪建模精度达到相同甚至更高的水平。     

Robust object tracking with RGBD-based sparse learning

Robust object tracking has been an important and challenging research area in the field of computer vision for decades. With the increasing popularity of affordable depth sensors, range data is widely used in visual tracking for its ability to provide robustness to varying illumination and occlusions. In this paper, a novel RGBD and sparse learning based tracker is proposed. The range data is integrated into the sparse learning framework in three respects. First, an extra depth view is added to the color image based visual features as an independent view for robust appearance modeling. Then, a special occlusion template set is designed to replenish the existing dictionary for handling various occlusion conditions. Finally, a depth-based occlusion detection method is proposed to efficiently determine an accurate time for the template update. Extensive experiments on both KITTI and Princeton data sets demonstrate that the proposed tracker outperforms the state-of-the-art tracking algorithms, including both sparse learning and RGBD based methods.     

基于三维重建的交通流量检测算法

在智能交通系统中 ,道路交通流量信息实时、有效的检测是交通信息系统的关键环节 .固定相机的视频图象检测法具有诸多优点 ,为此 ,提出了一个基于知识的视频图象交通流量检测系统 ,其中 ,车辆的分割和识别是视频检测法的核心 .根据车辆具有较大的运动惯性等运动规律 ,在短时间隔内 ,可以近似认为车辆运动为刚体匀速直线运动 .在这一条件下 ,将刚体上的运动点重投影到道路平面 ,则重投速度与该点的空间位置到路面的高度具有固定的比例关系 .运动特征采用具有较好定位精度的边缘特征 ,并拟合为直线进行运动跟踪匹配 .在识别过程中 ,先假定车辆的模型及其高度 ,然后再根据重投影速度 ,重建车辆的三维空间结构 ,进行基于知识规则的假设校验 .试验结果表明 ,该方法可以较好地解决车辆视频检测中的遮挡、粘连、阴影等情况     

基于双准则自适应融合的目标跟踪算法

针对单一评判准则较难适应复杂环境下的目标跟踪问题,提出了一种基于双评判准则自适应融合的跟踪算法。在该算法中,空间直方图被用作目标表示模型,候选目标与目标模板之间的相似度、以及候选目标与其邻近背景区域之间的对比度被作为目标评判双准则,而目标函数(或似然函数)则由两个准则的加权融合而成。算法是在粒子滤波框架下实现的目标搜索,并采用了模糊逻辑对相似度和对比度的权值进行自适应调节。对人、动物等多个挑战性运动目标的跟踪结果表明,与增量学习跟踪、ι₁跟踪等最新跟踪器相比,所提算法在处理目标的遮挡、形变、旋转以及表观变化方面的综合性能更好,其成功率和平均重叠率指标分别在80%和0.76以上。     

A real-time vehicle detection and a novel vehicle tracking systems for estimating and monitoring traffic flow on highways

Real-time highway traffic monitoring systems play a vital role in road traffic management, planning, and preventing frequent traffic jams, traffic rule violations, and fatal road accidents. These systems rely entirely on online traffic flow info estimated from time-dependent vehicle trajectories. Vehicle trajectories are extracted from vehicle detection and tracking data obtained by processing road-side camera images. General-purpose object detectors including Yolo, SSD, EfficientNet have been utilized extensively for real-time object detection task, but, in principle, Yolo is preferred because it provides a high frame per second (FPS) performance and robust object localization functionality. However, this algorithm’s average vehicle classification accuracy is below 57%, which is insufficient for traffic flow monitoring. This study proposes improving the vehicle classification accuracy of Yolo, and developing a novel bounding box (Bbox)-based vehicle tracking algorithm. For this purpose, a new vehicle dataset is prepared by annotating 7216 images with 123831 object patterns collected from highway videos. Nine machine learning-based classifiers and a CNN-based classifier were selected. Next, the classifiers were trained via the dataset. One out of ten classifiers with the highest accuracy was selected to combine to Yolo. This way, the classification accuracy of the Yolo-based vehicle detector was increased from 57% to 95.45%. Vehicle detector 1 (Yolo) and vehicle detector 2 (Yolo + best classifier), and the Kalman filter-based tracking as vehicle tracker 1 and the Bbox-based tracking as vehicle tracker 2 were applied to the categorical/total vehicle counting tasks on 4 highway videos. The vehicle counting results show that the vehicle counting accuracy of the developed approach (vehicle detector 2 + vehicle tracker 2) was improved by 13.25% and this method performed better than the other 3 vehicle counting systems implemented in this study.     

多模型协作的分块目标跟踪

为解决复杂场景下,基于整体表观模型的目标跟踪算法容易丢失目标的问题,提出一种多模型协作的分块目标跟踪算法.融合基于局部敏感直方图的产生式模型和基于超像素分割的判别式模型构建目标表观模型,提取局部敏感直方图的亮度不变特征来抵制光照变化的影响;引入目标模型的自适应分块划分策略以解决局部敏感直方图算法缺少有效遮挡处理机制的问题,提高目标的抗遮挡性;通过相对熵和均值聚类度量子块的局部差异置信度和目标背景置信度,建立双权值约束机制和子块异步更新策略,在粒子滤波框架下,选择置信度高的子块定位目标.实验结果表明,本文方法在复杂场景下具有良好的跟踪精度和稳定性.     

Moving vehicle detection,tracking and traffic parameter estimation from a satellite video: a perspective on a smarter city

ABSTRACT

Satellite remote sensing is undergoing a revolution in terms of sensors and temporal coverage. The possibility of acquiring earth’s surface video from space provides an opportunity to investigate broader applications of remote sensing. High-resolution spaceborne videos can become a vital factor in earth observation. Temporally continuous tracking of moving objects, i.e. vehicles, vessels, or even military equipment on Earth’s surface demands high spatial resolution satellite videos. Detecting moving vehicles in the urban areas from space video can lead governments to a new era of traffic monitoring. Satellite videos will find many applications in the field of traffic monitoring. In this article, first, moving vehicles are detected using background subtraction with 94.7% accuracy. Afterwards, vehicles’ trajectories, average velocities, dynamic velocities, and space-time diagram are estimated and trajectories are classified based on velocities. Finally, the total frame traffic density is computed.     

多特征重检测的相关滤波无人机视觉跟踪

无人机视觉跟踪是视觉跟踪未来应用的核心领域,其由于跟踪目标像幅较小、表 观不清且易受到无人机飞行姿态多变、飞行稳定性差等因素的影响而难以对目标进行鲁棒的跟 踪,特别是发生跟踪遮挡时,算法跟踪漂移后无法进行模型的更新。为提高无人机视频的跟踪 效果,提出一种多特征重检测跟踪方法。首先采用多特征融合的方式提高跟踪算法在无人机跟 踪特征的判别性。其次目标在出现遮挡时,扩大搜索区域,采用滑动窗口采样找到置信度最高 的目标区域并实现模型更新。通过一系列无人机视频实验结果表明,该算法在遇到遮挡问题时 具有较好的鲁棒性,能够提高无人机在目标跟踪过程中的准确性。     

Trajectory-based vehicle tracking at low frame rates

In smart cities, an intelligent traffic surveillance system plays a crucial role in reducing traffic jams and air pollution, thus improving the quality of life. An intelligent traffic surveillance should be able to detect and track multiple vehicles in real-time using only limited resources. Conventional tracking methods usually run at a high video-sampling rate, assuming that the same vehicles in successive frames are similar and move only slightly. However, in cost effective embedded surveillance systems (e.g., a distributed wireless network of smart cameras), video frame rates are typically low because of limited system resources. Therefore, conventional tracking methods perform poorly in embedded surveillance systems because of discontinuity of the moving vehicles in the captured recordings. In this study, we present a fast and light algorithm that is suitable for an embedded real-time visual surveillance system to detect effectively and track multiple moving vehicles whose appearance and/or position changes abruptly at a low frame rate. For effective tracking at low frame rates, we propose a new matching criterion based on greedy data association using appearance and position similarities between detections and trackers. To manage abrupt appearance changes, manifold learning is used to calculate appearance similarity. To manage abrupt changes in motion, the next probable centroid area of the tracker is predicted using trajectory information. The position similarity is then calculated based on the predicted next position and progress direction of the tracker. The proposed method demonstrates efficient tracking performance during rapid feature changes and is tested on an embedded platform (ARM with DSP-based system).     

基于仿射特征描述的鲁棒车辆遮挡检测方法

在交通场景下进行多目标跟踪时,往往会存在车辆之间的相互遮挡,使得基于视觉的车流量计算准确度受到严重影响。针对这一问题,提出一种基于仿射特征描述的鲁棒车辆遮挡检测方法,该方法利用车辆的角点特征,建立基于仿射变换的车辆特征描述模型,判断车辆之间是否存在遮挡现象,再对遮挡车辆进行特殊分割,保证了车辆跟踪以及车流量统计的准确性。通过50段5分钟连续视频,每个视频中出现车辆遮挡时段大于1分钟,遮挡频率25%,遮挡帧数大于500幅进行试验,算法的车流量统计的正确性达到95%,正确跟踪率达到98%,验证了该算法的有效性。     

Robust visual tracking and vehicle classification via sparse representation

In this paper, we propose a robust visual tracking method by casting tracking as a sparse approximation problem in a particle filter framework. In this framework, occlusion, noise, and other challenging issues are addressed seamlessly through a set of trivial templates. Specifically, to find the tracking target in a new frame, each target candidate is sparsely represented in the space spanned by target templates and trivial templates. The sparsity is achieved by solving an l1-regularized least-squares problem. Then, the candidate with the smallest projection error is taken as the tracking target. After that, tracking is continued using a Bayesian state inference framework. Two strategies are used to further improve the tracking performance. First, target templates are dynamically updated to capture appearance changes. Second, nonnegativity constraints are enforced to filter out clutter which negatively resembles tracking targets. We test the proposed approach on numerous sequences involving different types of challenges, including occlusion and variations in illumination, scale, and pose. The proposed approach demonstrates excellent performance in comparison with previously proposed trackers. We also extend the method for simultaneous tracking and recognition by introducing a static template set which stores target images from different classes. The recognition result at each frame is propagated to produce the final result for the whole video. The approach is validated on a vehicle tracking and classification task using outdoor infrared video sequences.