Motion detection in moving camera videos using background modeling and FlowNet

Real-time moving object detection is challenging for moving cameras due to the moving background. Many studies use homography matrix to compensate for global motion by warping the background model to the current frame. Then, the pixel difference between the current frame and the background model is used for background subtraction. Moving pixels are extracted by applying adaptive threshold and some post-processing techniques. On the other hand, deep learning-based dense optical flow can be efficient enough to extract the moving pixels, but it increases computational cost. This study proposes a method to enhance a classical background modeling method with deep learning-based dense optical flow. The main contribution of this paper is to propose a fusing algorithm for dense optical flow and background modeling approach. The background modeling methods are error-prone, especially with continuous camera movement, while the optical flow method alone may not always be efficient. Our hybrid method fuses both techniques to improve the detection accuracy. We propose a software architecture to run background modeling and dense optical flow methods in parallel processes. The proposed implementation approach significantly increases the method’s working speed, while the proposed fusion and combining strategy improve detection results. The experimental results show that the proposed method can run at high speed and has satisfying performance against the methods in the literature.     

基于光流法的多分辨率电子稳像算法

针对动载体摄像系统中视频序列受载体姿态运动及抖动的干扰而出现的不稳定现象,提出一种基于光流算法的多分辨率电子稳像算法。首先,通过划定有效的运动估算区域取代对整帧图像的计算以降低计算量;然后,利用基于光流算法的多分辨率分层运动估计快速并精确地计算出包含平移、旋转以及缩放运动的相邻帧间仿射变换参数;最后,采用固定帧补偿算法,利用求得的仿射变换参数,对图像进行运动补偿,消除或减轻图像序列帧间的随机抖动,达到稳像的目的。实验结果表明,针对包含运动目标的动态场景,该算法可以精确地检测出视频序列帧间平移、旋转以及缩放等复杂的抖动,水平和垂直方向的稳像精确度小于1pixel,保证视频序列的稳定输出,可应用于目标跟踪系统中。     

无人机视频图像运动目标检测算法综述

运动目标检测是实现目标跟踪、交通监控、行为分析等任务的基础。但在无人机获取的视频图像中,无人机运动、旋翼震动或外界风力等客观因素使图像出现较为明显的背景、光照等变化,会对运动目标的检测产生影响。因此,如何降低干扰、提高检测精度,让无人机在运动目标检测领域发挥作用在信息时代具有相当重要的意义。无人机视频图像的运动目标检测相比传统运动目标检测,检测思路基本一致,但干扰因素众多。本文以此为切入点,分类综述了适用于无人机视频图像运动目标检测的算法及其改进,主要包括运动估计算法、帧间差法、背景建模法、光流法等传统算法和近年出现的新型算法;通过对无人机运动状态的划分探讨比较了上述方法的优缺点及适用场景。帧间差法更适合处理无人机悬停状态的数据,背景建模法、光流法及新型算法对无人机悬停及巡航状态的数据均可处理;上述算法均不能很好解决光照变化造成误检、漏检现象。所以处理无人机视频数据时,要根据其运动信息及数据特点选择合适的算法,才能获得好的检测结果。     

较大尺度运动下的人体特征点跟踪算法研究

利用光流法可以对视频中运动目标进行特征点跟踪,当目标存在较大尺度运动时,光流法图像一致性假设难以满足,导致特征点跟踪丢失。针对此问题,提出了一种基于Lucas-Kanade(L-K)金字塔光流算法的运动人体特征点跟踪方法。首先,利用帧间差分法得到帧差图像序列,获取行人的运动区域;然后用尺度不变特征变换（SIFT）算法检测选定初始帧中的特征点;最后运用L-K金字塔光流算法跟踪这些特征点在后续帧中的位置。实验结果表明,该算法对较大尺度运动的特征点跟踪有很好的效果,提高了跟踪的准确性。      

基于三维递归搜索的多级运动估计视频帧率上转换算法

运动补偿插帧是目前主要的帧率上转换方法。为减小内插帧中的块效应,并降低运算量以满足实时高清视频应用,该文提出了一种基于3维递归搜索(3-D Recursive Search, 3-D RS)的多级块匹配运动估计视频帧率上转换算法。该算法将3-D RS与双向运动估计相结合,首先对序列中相邻帧进行由粗到精的三级运动估计,再利用简化的中值滤波器平滑运动矢量场,最后通过线性插值补偿得到内插帧。实验结果表明,与现有的运动补偿插帧算法相比,该算法内插帧的主、客观质量都有所提高,且算法复杂度低,有很强的实用性。     

Human centered perceptual adaptation for video coding

Traditional visual saliency based video compression methods try to encode the image with higher quality in the region of saliency. However, the saliency feature changes according to persons, viewpoints, and distances. In this paper, we propose to apply a technique of human centered perceptual computation to improve video coding in the region of human centered perception. To detect the region of interest (ROI) of human body, upper body, frontal face, and profile face, we construct Harr and histogram of oriented gradients features based combo of detectors to analyze a video in the first frame (intra-frame). From the second frame (inter-frame) onward, the optical flow image is computed in the ROI area of the first frame. The optical flow in human centered ROI is then used for macroblock (MB) quantization adjustment in H.264/AVC. For each MB, the quantization parameter (QP) is optimized with density value of optical flow image. The QP optimization process is based on a MB mapping model, which can be calculated by an inverse of the inverse tangent function. The Lagrange multiplier in the rate distortion optimization is also adapted so that the MB distortion at human centered region is minimized. We apply our technique to the H.264 video encoder to improve coding visual quality. By evaluating our scheme with the H.264 reference software, our results show that the proposed algorithm can improve the visual quality of ROI by about 1.01 dB while preserving coding efficiency.     

基于GPU并行运算的红外运动目标实时增强算法

在一些复杂场景中,红外目标容易受到背景杂波及噪声的干扰,因此难以被准确地检测和识别出来。提出了一种基于光流的红外运动目标增强算法,即利用运动目标与背景之间的速度场差异对目标进行增强处理.同时,对该算法进行了基于计算机图形处理器(Graphic Processor Unit,GPU)并行运算的优化,使其可以在线实时运行.与运动目标检测中常用的帧差法和背景差分法相比,本文算法具有更好的稳健性。由于对实际的红外视频进行了运动目标增强处理,该算法表现出了较好的增强效果和实时性能.     

基于光流和贝叶斯算法的目标跟踪研究

在本文中提出了一种方法来处理视频流中移动目标的跟踪,并且能够确定目标的移动速度。利用光流与贝叶斯方法在每一帧图像中检测对象,这种方法能够提高光流的检测性能,利用目标的质心像素位移来确定目标的移动距离,目标的速度是利用系列帧中目标移动的距离之间的帧数来计算的。实验表明,该方法能够有效地进行目标跟踪。     

视频图像中的运动目标检测

运动目标检测,是指从视频图像中将运动变化区域提取出来的检测技术,是图像处理技术的基础。在军事公安、交通管理、视频监控、医学检查等领域应用广泛。为了改进单独采用帧差法或背景减法进行运动目标检测时存在的不足,本文提出一种利用边缘信息的三帧差法与基于混合高斯模型的背景减法相结合的运动目标检测算法。该方法对视频图像中连续的三帧图像两两差分,对3个差分图像取均值,二值化,再经过形态学处理,并对中间帧进行Canny边缘提取,将二者进行"与"运算,即得到运动目标的边缘,用背景减法提取中间帧的前景,二值化,将其和目标的边缘进行"或"运算,经过形态学处理便可得到运动目标。实验结果表明,使用该方法目标检出率提高了9.7%~72.1%,误检率降低了0.090%~2.900%。这种二者相结合的方法相对于单一的检测算法能够有效、可靠地提取出运动目标。     

基于优选特征轨迹的全分辨率视频稳定

该文提出一种基于优选特征轨迹的视频稳定算法。首先,采用改进的Harris角点检测算子提取特征点,通过K-Means聚类算法剔除前景特征点。然后,利用帧间特征点的空间运动一致性减少错误匹配和时间运动相似性实现长时间跟踪,从而获取有效特征轨迹。最后,建立同时包含特征轨迹平滑度与视频质量退化程度的目标函数计算视频序列的几何变换集以平滑特征轨迹获取稳定视频。针对图像扭曲产生的空白区,由当前帧定义区与参考帧的光流作引导来腐蚀,并通过图像拼接填充仍属于空白区的像素。经仿真验证,该文方法稳定的视频,空白区面积仅为Matsushita方法的33%左右,对动态复杂场景和多个大运动前景均具有较高的有效性并可生成内容完整的视频,既提高了视频的视觉效果,又减轻了费时的边界修复任务。     

嵌入式下公共场所暴力行为视觉检测方法的研究与应用

根据暴力行为发生时往往存在肢体冲突,并伴随着身体全身或局部部位出现较剧烈运动的特点,本文提出了一种基于运动图像的公共场所暴力行为自动识别方法,并能够移植到嵌入式设备中以便于分布式智能监控。首先,采用Lucas-Kanade(LK)光流法获得相邻两帧监控图像间的光流场,通过光流分析法确定各个运动目标的运动状态,并提取它们的光流特征值、光流速度和方向;其次,对各个运动目标的光流特征值、光流速度与方向进行统计分析,以掌握各个运动目标的运动趋势;最后,结合光流场变化情况及运动目标聚集状态判断是否存在暴力行为。实验结果表明,该嵌入式系统识别准确性好、实时性强、可靠性高。     

Image stabilization and registration for tracking cells in the microvasculature

We propose a registration system to be used for tracking cells in intravital video microscopy that 1) stabilizes jitter-the undesired translational displacement of frames due to respiratory movement, etc., and 2) registers frames in a moving field of view (FOV) to allow for cell tracking over an extended range. For the first time, tracking of rolling leukocytes in vivo over a moving FOV is demonstrated. In a fixed FOV, stable background regions are located using a morphological approach. Template subregions are then selected from the stable regions and matched to corresponding locations in a reference frame. We show the effectiveness of the stabilization algorithm by using an active contour to track 15 leukocytes previously untrackable due to jitter. For 30 fixed FOV sequences containing rolling leukocytes, the resulting root-mean-square error (RMSE) is less than 0.5 microm. To align frames in a moving FOV, we present a modified correlation approach to estimate the common region between two consecutive fixed FOVs. We correlate the overlapping regions of the initial frame of the current fixed FOV and the final frame of the previous fixed FOV to register the images in the adjoining moving FOV. The RMSE of our moving FOV registration technique was less than 0.6 mmicrom. In 10 sequences from different venules, we were able to track 11 cells using an active contour approach over moving FOVs.     

基于粒子滤波的视频图像目标遮挡算法研究

基于粒子滤波的视频图像目标遮挡算法是当前的一个热门研究领域.在对于视频图像目标跟踪方面,综合运用了多种算法进行检测和跟踪,详细分析光流法、帧间差分法、背景差分法和视频图像目标特征的提取,并在最后对帧间差分的算法进行了改进.通过实验证明,采用基于粒子滤波的视频图像目标遮挡算法能够更加有效地解决对跟踪目标的准确判断.     

基于多帧边缘差异的视频运动对象的分割与跟踪算法

从视频场景中分割和跟踪感兴趣的视频对象对于MPEG-4等基于对象的视频编码来说是关键性的技术之一。针对目前大部分视频对象分割和追踪算法相当复杂但仍不能有效地去除背景噪声的问题,该文提出用于分割和跟踪视频运动对象的一种基于多帧边缘差异的算法。该算法利用一组帧的边缘差异来提取运动对象区域,通过聚类方法去除背景像素点,利用形态学算子得到对象分割模板,同时通过建立前帧感兴趣对象与当前帧运动对象的帧间向量跟踪当前帧的感兴趣视频对象。不同标准视频测试序列的测试结果表明,该算法能够实现对感兴趣的视频运动对象更为精确、快速和有效地分割和跟踪。     

动态背景下的光流场计算方法

通过计算光流场来检测场景中的运动目标是计算机视觉中非常重要的研究课题,而光流场计算的精度直接关系到目标检测的准确性。针对实际拍摄的视频中由于背景存在运动而导致光流场中运动目标不突出的情况,提出了一种基于分块积分投影配准算法的光流场计算方法。首先利用提出的分块积分投影配准算法得到图像背景的运动参数,然后对背景进行运动补偿,再利用L-K算法求取运动补偿后图像中有效区域的光流场。通过真实视频对算法进行验证,并将结果与经典的L-K算法结果进行了对比。对比结果显示:本文所提算法计算得到的光流场中运动目标更加突出,算法效果较好。     

HOS运动目标分割算法在视频监控中的应用

为了提高视频监控中运动目标分割的速度和准确度,研究并实现了一种基于高阶统计量HOS(HigherOrder Statistics)的分割算法.首先根据HOS假设检验处理帧差图,判定像素点是否属于运动区域,阈值通过灰度共生矩阵获得,考虑了背景纹理的慢变化.然后,用矩形框聚类法大致确定运动目标的范围,在该范围内使用形态运算法和首尾扫描法去除空洞.最后,使用模板相与法获得帧图像的运动目标模板,从原图像中分割运动区域.算法采用了由粗到精的分析策略,实验表明,是一种快速稳健的算法.     

Segmentation and tracking of moving objects for content-based videocoding

To enable content-based functionalities in video coding, a decomposition of the scene into physical objects is required. Such objects are normally not characterised by homogeneous colour, intensity, or optical flow. Therefore, conventional techniques based on these low-level features cannot perform the desired segmentation. The authors address segmentation and tracking of moving objects and present a new video object plane (VOP) segmentation algorithm that extracts semantically meaningful objects. A morphological motion filter detects physical objects by identifying areas that are moving differently from the background. A new filter criterion is introduced that measures the deviation of the estimated local motion from the synthesised global motion. A two-dimensional binary model is derived for the object of interest and tracked throughout the sequence by a Hausdorff object tracker. To accommodate for rotations and changes in shape, the model is updated every frame by a two-stage method that accounts for rigid and non-rigid moving parts of the object. The binary model then guides the actual VOP extraction, whereby a novel boundary post-processor ensures high boundary accuracy. Experimental results demonstrate the performance of the proposed algorithm     

基于帧间差分和金字塔LK光流法的运动目标检测

针对用于运动目标检测的光流算法存在处理复杂、计算量大等问题,提出一种帧间差分算法和金字塔LK光流法相结合的运动目标检测方案.该方法先对视频图像进行帧间差分处理,得到图像的运动区域,再对该运动区域进行金字塔LK光流计算,减少了计算区域,提高目标检测的速度.最后在搭建的视觉避障平台上使用LabVIEW语言进行算法程序验证,实验结果证明了算法的有效性.     

Optical flow estimation using temporally oversampled video.

Recent advances in imaging sensor technology make high frame-rate video capture practical. As demonstrated in previous work, this capability can be used to enhance the performance of many image and video processing applications. The idea is to use the high frame-rate capability to temporally oversample the scene and, thus, to obtain more accurate information about scene motion and illumination. This information is then used to improve the performance of image and standard frame-rate video applications. This paper investigates the use of temporal oversampling to improve the accuracy of optical flow estimation (OFE). A method for obtaining high accuracy optical flow estimates at a conventional standard frame rate, e.g., 30 frames/s, by first capturing and processing a high frame-rate version of the video is presented. The method uses the Lucas-Kanade algorithm to obtain optical flow estimates at a high frame rate, which are then accumulated and refined to estimate the optical flow at the desired standard frame rate. The method demonstrates significant improvements in OFE accuracy both on synthetically generated video sequences and on a real video sequence captured using an experimental high-speed imaging system. It is then shown that a key benefit of using temporal oversampling to estimate optical flow is the reduction in motion aliasing. Using sinusoidal input sequences, the reduction in motion aliasing is identified and the desired minimum sampling rate as a function of the velocity and spatial bandwidth of the scene is determined. Using both synthetic and real video sequences, it is shown that temporal oversampling improves OFE accuracy by reducing motion aliasing not only for areas with large displacements but also for areas with small displacements and high spatial frequencies. The use of other OFE algorithms with temporally oversampled video is then discussed. In particular, the Haussecker algorithm is extended to work with high frame-rate sequences. This extension demonstrates yet another important benefit of temporal oversampling, which is improving OFE accuracy when brightness varies with time.     

背景渐变的视频对象分割算法研究及实现

提出了一种有效的背景渐变的视频对象分割算法.首先将前一帧分成前景和背景两部分,然后采用灰度投影匹配算法对当前帧进行全局运动估计和补偿,将当前帧与上一帧进行差分运算,便可得到差分图像.通过对差分图像进行二值化处理,得到运动模板并与前景信息进行相与计算,再结合当前帧信息便可得到运动目标.在TI公司的TMS320DM642芯片上验证了该算法,实验结果表明该算法不仅对亮度变化和环境变化具有鲁棒性,而且可独立、精确地分割出运动目标.