Abnormal crowd behavior detection using size-adapted spatio-temporal features

Abnormal crowd behavior detection is an important research issue in computer vision. However, complex real-life situations (e.g., severe occlusion, over-crowding, etc.) still challenge the effectiveness of previous algorithms. Recently, the methods based on spatio-temporal cuboid are popular in video analysis. To our knowledge, the spatio-temporal cuboid is always extracted randomly from a video sequence in the existing methods. The size of each cuboid and the total number of cuboids are determined empirically. The extracted features either contain the redundant information or lose a lot of important information which extremely affect the accuracy. In this paper, we propose an improved method. In our method, the spatio-temporal cuboid is no longer determined arbitrarily, but by the information contained in the video sequence. The spatio-temporal cuboid is extracted from video sequence with adaptive size. The total number of cuboids and the extracting positions can be determined automatically. Moreover, to compute the similarity between two spatio-temporal cuboids with different sizes, we design a novel data structure of codebook which is constructed as a set of two-level trees. The experiment results show that the detection rates of false positive and false negative are significantly reduced. Keywords: Codebook, latent dirichlet allocation (LDA), social force model, spatio-temporal cuboid.     

基于多尺度时间递归神经网络的人群异常检测

如何在人群密度大、变化快、存在大量遮挡的密集场景中实现可靠的人群事件检测,是领域研究的难点和热点.在密集场景时空建模的基础上提出了一种基于多尺度时间递归神经网络的人群异常事件检测和定位方法.首先对人群场景进行网格化划分,并利用多尺度光流直方图对每个网格的人群动态进行刻画;然后,连接各个局部的人群动态获得整体的人群动态,实现整体人群动态的时间序列建模;最后,利用多尺度时间递归神经网络实现异常事件的检测和定位.其中,多尺度隐含层实现了密集场景中不同规模相邻网格之间的空间联系,节点间的反馈关系则为时间维度上的关系表达提供了有效方案.与多种代表性算法的对比实验,验证了本方法的有效性.     

Tracking pedestrians using local spatio-temporal motion patterns in extremely crowded scenes

Tracking pedestrians is a vital component of many computer vision applications, including surveillance, scene understanding, and behavior analysis. Videos of crowded scenes present significant challenges to tracking due to the large number of pedestrians and the frequent partial occlusions that they produce. The movement of each pedestrian, however, contributes to the overall crowd motion (i.e., the collective motions of the scene's constituents over the entire video) that exhibits an underlying spatially and temporally varying structured pattern. In this paper, we present a novel Bayesian framework for tracking pedestrians in videos of crowded scenes using a space-time model of the crowd motion. We represent the crowd motion with a collection of hidden Markov models trained on local spatio-temporal motion patterns, i.e., the motion patterns exhibited by pedestrians as they move through local space-time regions of the video. Using this unique representation, we predict the next local spatio-temporal motion pattern a tracked pedestrian will exhibit based on the observed frames of the video. We then use this prediction as a prior for tracking the movement of an individual in videos of extremely crowded scenes. We show that our approach of leveraging the crowd motion enables tracking in videos of complex scenes that present unique difficulty to other approaches.     

基于C3D-GRNN模型的人群异常行为识别算法

针对传统人群行为识别算法受人工主观因素影响较大等问题,综合三维卷积神经网络（C3D）与广义回归神经网络（GRNN）的优势和特点,提出并实现了基于C3D-GRNN模型的人群异常行为识别算法。该算法首先采用ViBe算法确定运动目标区域,然后通过改进C3D网络提取目标的HOG时空特征,再加入GRNN层进行分类训练,最后根据训练好的C3D-GRNN模型完成人群异常行为的识别与分类。实验结果表明：所提算法的HOG时空特征能够明显提升对人群行为的表达能力,减少了特征提取工作量,并且该方法的准确度和鲁棒性均高于支持向量机等其他同类方法,为小样本数据集的分类问题提供解决新思路,具有较高的应用价值。     

Crowd flux analysis and abnormal event detection in unstructured and structured scenes

This paper presents a spatio-temporal grid-based framework to deal with the complexity of structured and unstructured motion flows that can effectively group optical flows in the field of view into crowds. This approach utilizes motion flows of the features based on a grid in a scene. In order to detect abnormal events in crowded scenes, the proposed method measures motion features including the speed and direction of moving objects based on a spatio-temporal grid-based approach for flow representation. Experiments have been conducted on several different videos in three domains that are crosswalks, escalators, and highways. To evaluate and compare the performance of our method to other methods, ROC curves are plotted which take into consideration both detection rate and false alarm rate for multiple threshold values.     

基于群体性行为模型的人群分析及其应用

基于视频分析的人群监控,涉及到获取人群行为和数量,这在智能监控领域具有重要的现实价值。本文建立基于运动特征的群体性行为模型,挖掘复杂人群场景中的群体行为,用于人群行为和数量的分析。群体性行为模型是一种主题模型（LDA）,通过样本学习,可以获得描述不同群体行为的特征集,用于人群分析。实验中,将群体性行为模型应用于挖掘监控场景下的不同人群行为及其特征集,并使用人工神经网络完成人数统计,统计正确率达到92.35%。     

利用角点历史信息的异常行为识别算法

针对视频监控场景中的异常行为事件,如突然的奔跑、人群的异常聚集等现象,提出一种利用角点运动历史图策略的行为识别算法,即首先通过角点提取算法进行场景角点提取;然后通过时间累积获取角点的历史图,通过角点历史图将场景中的角点划分为静态角点和动态角点;最后通过动态角点分析完成监控场景异常行为分析识别。新算法充分利用了图像的时空信息,并且克服了场景光照影响,增强了异常行为检测与识别的准确性。通过真实场景实验可以看出,新算法能够对不同监控场景的异常行为进行准确检测,并且其检测速度快,满足实际应用需求。     

结合流形密度的聚集行为模式分割算法

目的 在视频监控和人群模式行为理解的重要应用中,识别分割场景中的集体行为仍然是一个极具挑战性的问题。在这项研究中,提出一种基于流形密度的集体聚类算法,能够识别具有任意形状和不同密度条件下的集体行为的局部和全局模式。方法 受群体运动行为的流形拓扑结构启发,首先提出一种新的流形距离度量方式用于挖掘群体运动的深层行为模式。进一步定义了集体聚集密度的概念,并通过基于聚集密度的聚类算法识别具有局部一致性行为的群组,这种策略更适用于识别具有任意形状的聚类。同时考虑到子群组之间的复杂交互作用,引入层次聚集合并算法得到全局集体行为模式,可以有效地表征全局一致性关系。结果 针对不同情况下的复杂场景,本文算法在集体视频监控数据集下的实验结果表明了其有效性和鲁棒性,相比于传统的聚类方法和标准经典算法,以平均误差（AD）和方差（VAR）作为评价指标来评价算法性能,本文方法将识别分割聚集行为群组的误差率结果控制在了0.81和0.99以内,相比许多经典方法有较大提升。同时在具有复杂流形结构及任意密度条件下的人群场景中能够取得精确有效的识别结果,解决了经典方法在该特殊场景下存在的缺点。结论 本文针对已有方法在流形结构场景识别集体行为流向缺乏精确性和稳定性的描述和分析这一问题,提出了基于流形密度的群组聚集聚类识别算法,在多个复杂真实视频数据集中进行实验,证明了所提方法的有效性,并相比于已有方法具有更高的识别精度。     

基于智能监控的中小人群异常行为检测

针对人群异常行为检测实时性较差、分类算法识别率不高、特征量较少的问题,提出一种基于智能监控的中小人群异常行为检测算法。首先,利用快速群体密度检测算法,提取人群数量变化信息;其次,利用改进的Lucas-Kanande光流法提取视频中人群的平均动能、人群方向熵、人群距离势能;最后,利用极限学习机(ELM)算法对人群行为进行分类。使用UMN公共数据集进行测试,ELM算法对中小人群异常行为分析比中高密度人群异常行为检测算法和基于KOD能量特征的群体异常行为检测算法识别率分别高出7.13个百分点和5.89个百分点,并且人数密度估计部分平均每帧图像处理耗时相比中高密度人群异常行为检测算法减少了106 ms(近1/3)。实验结果表明:基于智能监控的中小人群异常行为检测算法能有效提高异常帧识别率和实时性。     

融合门控自注意力机制的生成对抗网络视频异常检测

目的 视频异常行为检测是当前智能监控技术的研究热点之一,在社会安防领域具有重要应用。如何通过有效地对视频空间维度信息和时间维度信息建模来提高异常检测的精度仍是目前研究的难点。由于结构优势,生成对抗网络目前广泛应用于视频异常检测任务。针对传统生成对抗网络时空特征利用率低和检测效果差等问题,本文提出一种融合门控自注意力机制的生成对抗网络进行视频异常行为检测。方法 在生成对抗网络的生成网络U-net部分引入门控自注意力机制,逐层对采样过程中的特征图进行权重分配,融合U-net网络和门控自注意力机制的性能优势,抑制输入视频帧中与异常检测任务不相关背景区域的特征表达,突出任务中不同目标对象的相关特征表达,更有效地针对时空维度信息进行建模。采用LiteFlownet网络对视频流中的运动信息进行提取,以保证视频序列之间的连续性。同时,加入强度损失函数、梯度损失函数和运动损失函数加强模型检测的稳定性,以实现对视频异常行为的检测。结果 在CUHK （Chinese University of Hong Kong） Avenue、UCSD （University of California,San Diego） Ped1和UCSD Ped2等视频异常事件数据集上进行实验。在CUHK Avenue数据集中,本文方法的AUC （area under curve）为87.2%,比同类方法高2.3%;在UCSD Ped1和UCSD Ped2数据集中,本文方法的AUC值均高于同类其他方法。同时,设计了4个消融实验并对实验结果进行对比分析,本文方法具有更高的AUC值。结论 实验结果表明,本文方法更适合视频异常检测任务,有效提高了异常行为检测任务模型的稳定性和准确率,且采用视频序列帧间运动信息能够显著提升异常行为检测性能。     

Hierarchical crowd analysis and anomaly detection

ObjectiveThis work proposes a novel approach to model the spatiotemporal distribution of crowd motions and detect anomalous events.MethodsWe first learn the regions of interest (ROIs) which inform the behavioral patterns by trajectory analysis with Hierarchical Dirichlet Processes (HDP), so that the main trends of crowd motions can be modeled. Based on the ROIs, we then build a series of histograms both on global and local levels as the templates for the observed movement distribution, which statistically describes time-correlated crowd events. Once the template has been built hierarchically, we import real data containing the discrete trajectory observations from video surveillance and detect abnormal events for individuals and for crowds.ResultsExperimental results show the effectiveness of our approach, which is able to analyze and extract the crowd motion information from observed trajectory dataset, and achieve the anomaly detection at the hierarchical levels.ConclusionThe proposed hierarchical approach can learn the moving trends of crowd both in global and local area and describe the crowd behaviors in statistical way, which build a template for pedestrian movement distribution that allows for the detection of time-correlated abnormal crowd events.     

融入情绪模型的人群实时路径规划

目的 人群仿真中高效性和逼真性是群体路径规划的关键问题。已有人群路径规划算法忽略了人群情绪造成的路径选择差异,在大规模实时仿真中仍存在一定的局限性。方法 提出一种融入情绪模型的人群实时路径规划算法(EPP)。使用人格特征理论对人群的情绪进行建模并设定不同情绪所具有的路径选择偏好。在路径建模阶段,通过单次搜索建立全局有向导航图,确定全局层面的可行路径;在路径搜索阶段,提出以期望时间最短为原则的路径规划目标函数,采用局部搜索策略为个体实时规划一条最优或次优的路径。结果 EPP算法可有效地仿真不同场景下大规模人群的路径选择现象;与已有工作的仿真效果和量化指标对比说明了该算法的有效性和高效性;通过不同情绪状态下人群路径选择差异的讨论以及在不同人群运动模型的兼容性实验进一步说明了该算法的健壮性。结论 本文算法具有良好的高效性和健壮性,适用于不同场景下大规模人群路径规划的相关应用。     

An efficient subsequence search for video anomaly detection and localization

This paper presents a novel framework for anomaly event detection and localization in crowded scenes. For anomaly detection, one-class support vector machine with Bayesian derivation is applied to detect unusual events. We also propose a novel event representation, called subsequence, which refers to a time series of spatial windows in proximity. Unlike recent works encoded an event with a 3D bounding box which may contain irrelevant information, e.g. background, a subsequence can concisely capture the unstructured property of an event. To efficiently locate anomalous subsequences in a video space, we propose the maximum subsequence search. The proposed search algorithm integrates local anomaly scores into a global consistent detection so that the start and end of an abnormal event can be determined under false and missing detections. Experimental results on two public datasets show that our method is robust to the illumination change and achieve at least 80% localization rate which approximately doubles the accuracy of recent works. This study concludes that anomaly localization is crucial in finding abnormal events.     

基于多层BP神经网络和无参数微调的人群计数方法

针对大部分现有的人群计数方法被应用到新的场景时性能下降的问题,在多层BP神经网络框架下,提出一种具有无参数微调的人群计数方法。首先,从训练图像中裁切图像块,将获得的相似尺度的行人作为人群BP神经网络模型的输入;然后,BP神经网络模型通过学习预测密度图,得到了一个具有代表性的人群块;最后,为了处理新场景,对训练好的BP神经网络模型进行目标场景微调,可追求有相同属性的样本,包括候选块检索和局部块检索。实验数据集包括PETS2009数据集、UCSD数据集和UCF_CC_50数据集。这些场景的实验结果验证了提出方法的有效性。相比于全局回归计数法和密度估计计数法,提出的方法在平均绝对误差和均方误差方面均有较大优势, 消除了场景间区别和前景分割的影响。     

基于多尺度特征融合的人群密度检测

基于深度学习的人群密度检测算法取得了巨大进步,但该算法在实际复杂场景中的检测准确性和鲁棒性还有很大的提升空间.复杂场景下目标尺度不一致和背景信息干扰等因素使得人群密度检测成为一项具有挑战性的任务.针对该问题,提出了一种基于多尺度特征融合的人群密度检测网络.该网络首先利用不同分辨率图像并行交互提取人群粗细粒度特征,并引入多层次特征融合机制,以充分利用多层尺度信息.其次采用空间和通道注意力机制突出人群特征权重,聚焦感兴趣的人群,降低背景信息干扰,生成高质量密度图.实验结果表明,在多个典型的公共数据集上与具有代表性的人群密度检测方法相比,多尺度特征融合的人群密度检测网络具有良好的准确性和鲁棒性.     

Abnormality detection in crowd videos by tracking sparse components

Abnormality detection in crowded scenes plays a very important role in automatic monitoring of surveillance feeds. Here we present a novel framework for abnormality detection in crowd videos. The key idea of the approach is that rarely or sparsely occurring events correspond to abnormal activities, while the regularly or commonly occurring events correspond to the normal activities. Each input video is represented using feature matrices that capture the nature of activity taking place while maintaining the spatial and temporal structure of the video. The feature matrices are decomposed into their low-rank and sparse components where sparse component corresponds to the abnormal activities. The approach does not require any explicit modeling of crowd behavior or training, but the information from training data can be seamlessly incorporated if it is available. The estimation is further improved by ensuring temporal and spatial coherence of sparse component across the videos using a Kalman filter-like framework. This not only results in reduction of outliers and noise but also fills missing regions in the sparse component. Localization of the anomalies is obtained as a by-product of the proposed approach. Evaluation on the UMN and UCSD datasets and comparisons with several state-of-the-art crowd abnormality detection approaches shows the effectiveness of the proposed approach. We also show results on a challenging crowd dataset created as part of this effort, with videos downloaded from the web.     

基于Faster R-CNN的密集人群检测算法

为提高拥挤场景下的人群检测准确率,提出一种基于改进Faster R-CNN的密集人群检测算法。首先,在特征提取阶段添加空间与通道注意力机制,使用加强的双向特征金字塔网络（S-BiFPN）替代原网络中的多尺度特征金字塔（FPN）,使网络对重要特征进行自主学习并加强对图像深层特征的提取;其次,引入多实例预测（MIP）算法对实例进行预测,以避免模型对拥挤场景下的目标造成漏检;最后,对模型中的非极大值抑制（NMS）进行优化,并额外增设一个交并比（IoU）阈值,以对检测结果的干扰项进行精确抑制。在开源的密集人群检测数据集上进行测试的结果显示,相较于原Faster R-CNN算法,所提算法的平均精度（AP）提升5.6%,Jaccard指数值提升3.2%。所提算法具有较高检测精度和稳定性,可以满足密集场景人群检测的需求。     

基于轨迹分段LDA主题模型的视频异常行为检测方法

基于目标轨迹的异常行为检测算法忽略了轨迹内部信息,容易导致异常检测虚警率偏高。为解决该问题,提出一种基于轨迹分段主题模型的视频异常行为检测方法。首先将目标原始轨迹根据轨迹转角分段,然后采用分段量化的方式提取轨迹片段中包含的行为特征信息,接着通过潜在狄利克雷分配(LDA)主题模型建模发掘目标轨迹之间的时空关系,最后通过学习所构建的模型并结合贝叶斯理论进行行为模式分析和异常行为检测。分别对两个视频场景进行了目标行为模式分析和异常行为检测的仿真实验,检测出了场景内多种异常行为模式。实验结果表明,通过结合轨迹分段与LDA主题模型,该算法能够充分挖掘目标轨迹内部的行为特征信息,识别多种异常行为模式,并且能提高对异常行为检测的准确率。     

基于KOD能量特征的群体异常行为识别

在群体异常行为识别过程中, 针对传统特征易受目标遮挡影响导致其对群体行为的弱描述性问题, 提出一种基于KOD（kinetic orientation distance）能量特征的群体异常行为识别方法。该能量特征忽略群体中相互遮挡的个体的局部特征, 从群体行为整体上分别根据群体的运动剧烈程度、群体运动方向一致性和群体中个体的相对位置定义并提取群体动能、方向势能和距离势能构成群体行为高层KOD能量特征, 以此描述群体的运动状态变化, 最后通过构建隐马尔可夫模型实现群体异常行为检测及类型识别。在PETS和UMN公共数据集上进行实验并与传统光流特征进行对比, 实验结果表明, 使用KOD能量特征能够有效地检测出群体异常行为并识别出其类型, 且能够达到92%的准确率。