基于运动矢量多级分析的视频全局运动估计

基于运动矢量场的视频全局运动估计相较于基于像素的估计方法具有较低的计算复杂度,因而广泛应用于视频分割及视频压缩等领域中。然而噪声和前景目标等外点区域的存在,降低了全局运动估计的准确性。为了提高全局运动估计的准确度,该文提出一种基于运动矢量多级分析的全局运动估计算法,该算法根据局部运动与全局运动的运动特性差异自适应地滤除前景目标区域,由邻域矢量间相似性度量检测出纹理平滑周期区域,最后滤除孤立的噪声区域,由滤波得到的内点区域求解全局运动参数。实验结果表明,该方法能有效地滤除外点区域,提高全局运动估计的准确性。     

动态背景下融合运动线索和颜色信息的视频目标分割算法

针对现有动态背景下目标分割算法存在的局限性,提出了一种融合运动线索和颜色信息的视频序列目标分割算法。首先,设计了一种新的运动轨迹分类方法,利用背景运动的低秩特性,结合累积确认的策略,可以获得准确的运动轨迹分类结果;然后,通过过分割算法获取视频序列的超像素集合,并计算超像素之间颜色信息的相似度;最后,以超像素为节点建立马尔可夫随机场模型,将运动轨迹分类信息以及超像素之间颜色信息统一建模在马尔可夫随机场的能量函数中,并通过能量函数最小化获得每个超像素的最优分类。在多组公开发布的视频序列中进行测试与对比,结果表明,本文方法可以准确分割出动态背景下的运动目标,并且较传统方法具有更高的分割准确率。     

Temporal image sequence prediction using motion field interpolation

A new method for motion-compensated temporal prediction of image sequences is proposed. Motion vector fields in natural scenes should possess two basic properties. First, the field should be smoothly varying within moving objects to compensate for nonrigid or rotational motion, and scaling of objects. Second, the field should be discontinuous along the boundaries of the objects. In the proposed method the motion vector field is modelled using finite element methods and interpolated using adaptive interpolators to satisfy the above-stated requirements. This is particularly important when only very sparse estimates of motion vector fields are available in the decoder due to bit-rate constraints limiting the amount of overhead information that can be transmitted. The proposed prediction method can be applied for low-bit-rate video coding in conventional codecs based on motion-compensated prediction and transform coding, as well as in model-based codecs. The performance of the proposed method is compared with standard motion-compensated prediction based on block matching. It is shown that for simple video telephony scenes a reduction of more than 30% in the energy of the prediction error can be achieved with an unchanged number of transmitted motion vectors and with only a modest increase in computational complexity. When implemented in an H.261 codec the new prediction method can improve the peak SNR 1–2 dB producing a significant visual improvement.     

基于模糊聚类的运动对象分割算法研究

为了从视频序列中分割出完整的、一致的运动视频对象,该文使用基于模糊聚类的分割算法获得组成对象边界的像素,从而提取对象。该算法首先使用了当前帧以及之前一些帧的图像信息计算其在小波域中不同子带的运动特征,并根据这些运动特征构造了低分辨率图像的运动特征矢量集;然后,使用模糊C-均值聚类算法分离出图像中发生显著变化的像素,以此代替帧间差图像,并利用传统的变化检测方法获得对象变化检测模型,从而提取对象;同时,使用相继两帧之间的平均绝对差值大小确定计算当前帧运动特征所需帧的数量,保证提取视频对象的精确性。实验结果证明该方法对于分割各种图像序列中的视频对象是有效的。     

一种面向对象的视频分割算法

该文讨论面向对象编码的视频分割算法。由于数学形态学工具能够很好地处理诸如大小、形状、对比度和连通性等对图像分割非常重要的特征,这种技术常用来对图像进行帧内分割,得到一些具有某种相似性的区域。然后利用运动信息进行区域合并。为了实现区域合并,该文提出一种新方法来判定局部运动。设计了一个分割细化步骤,对区域边界点进行再判决,可以得到更好的结果。实验结果表明,该文提出的方法对于平稳背景和运动背景中的视频对象分割都是有效的。     

基于运动信息和标记多尺度分水岭的运动目标检测算法

针对对称差分法检测目标时容易产生空洞以及当目标运动速度较慢或尺寸较小时易出现漏检等现象,提出了一种基于运动信息和标记多尺度分水岭的运动目标检测算法。首先将用高频强调滤波等处理的视频图像进行差分,再运用高阶统计运动检测算法检测出差分图像中运动目标的大概运动区域,经后处理得到运动目标的初始二值掩膜;并应用初始二值掩膜得到用于标记的前景与背景标识,用该标识修正当前帧的多尺度形态学梯度图像;最后进行分水岭分割,得到具有精确边界的运动目标。实验结果表明该方法能对运动目标进行准确检测,继承了变化检测和分水岭算法速度快的特点,克服了分水岭算法易产生过分割的缺点,且具有良好的鲁棒性。     

Robust bilayer video segmentation by adaptive propagation of global shape and local appearance

Segmenting semantic objects of interest from video has long been an active research topic, with a wide range of potential applications. In this paper, we present a bilayer video segmentation method robust to abrupt motion and change in appearance for both the foreground and background. Specifically, based on a few manually segmented keyframes, the proposed method propagates the global shape of the foreground as priors to adjacent frames by applying branch-and-mincut [1], which jointly estimates what is optimal among a set of shapes along with its pose and the corresponding segmentation in the current image. Based on this preliminary segmentation we determine two types of local regions likely to have erroneous results, and apply a probabilistic framework where shape and appearance cues are adaptively emphasized for local refinement. With each successive frame segmentation, the set of shapes applied as priors are incrementally updated. Experimental results support the robustness of the proposed method for obstacles such as background clutter, motion, and appearance changes, from only a small number of user segmented keyframes.     

Segmentation and tracking of moving objects for content-based videocoding

To enable content-based functionalities in video coding, a decomposition of the scene into physical objects is required. Such objects are normally not characterised by homogeneous colour, intensity, or optical flow. Therefore, conventional techniques based on these low-level features cannot perform the desired segmentation. The authors address segmentation and tracking of moving objects and present a new video object plane (VOP) segmentation algorithm that extracts semantically meaningful objects. A morphological motion filter detects physical objects by identifying areas that are moving differently from the background. A new filter criterion is introduced that measures the deviation of the estimated local motion from the synthesised global motion. A two-dimensional binary model is derived for the object of interest and tracked throughout the sequence by a Hausdorff object tracker. To accommodate for rotations and changes in shape, the model is updated every frame by a two-stage method that accounts for rigid and non-rigid moving parts of the object. The binary model then guides the actual VOP extraction, whereby a novel boundary post-processor ensures high boundary accuracy. Experimental results demonstrate the performance of the proposed algorithm     

基于多帧边缘差异的视频运动对象的分割与跟踪算法

从视频场景中分割和跟踪感兴趣的视频对象对于MPEG-4等基于对象的视频编码来说是关键性的技术之一。针对目前大部分视频对象分割和追踪算法相当复杂但仍不能有效地去除背景噪声的问题,该文提出用于分割和跟踪视频运动对象的一种基于多帧边缘差异的算法。该算法利用一组帧的边缘差异来提取运动对象区域,通过聚类方法去除背景像素点,利用形态学算子得到对象分割模板,同时通过建立前帧感兴趣对象与当前帧运动对象的帧间向量跟踪当前帧的感兴趣视频对象。不同标准视频测试序列的测试结果表明,该算法能够实现对感兴趣的视频运动对象更为精确、快速和有效地分割和跟踪。     

运动视频对象的时空联合检测技术

提出了一种具有全局运动的视频运动对象时空联合检测算法。针对传统时间分割使用主观固定阈值的缺点,采用了对差分图像进行噪声参数自适应学习的算法获取自动阈值,并利用形态学运算获取修正的时间分割模板;考虑传统分水岭空间分割的不足,提出了基于人眼视觉特征的改进分水岭算法,包括基于形态重建滤波的图像降噪、形态梯度变换以及基于韦伯感知原理的视同灰度非线性变换,有效地解决了过分割问题;对时、空间分割结果进行信息融合处理,从而得到完整的运动对象。仿真实验结果表明,本文算法可以快速准确地分割视频运动对象。     

Automatic moving object and background separation

In this paper, we propose a segmentation method of reduced computational complexity aimed at separating the moving objects from the background in a generic video sequence. This task may be accomplished at the coder site to support the functionalities foreseen by new multimedia scenarios, and in particular the content-based functionalities focused by the MPEG-4 activity, allowing the user to access and decode single objects of a video sequence. The proposed algorithm discriminates between background and foreground by means of a higher-order statistics (HOS) significance test performed on a group of inter-frame differences, followed by a motion detection phase, producing a binary segmentation map. The HOS threshold is adaptively changed, based on the estimated background activity and on the potential presence of slowly moving objects. The map is refined by a final regularization stage implemented by means of a cascade of morphological filters. The algorithm performance were tested through the wide experimental activity carried out during the ISO MPEG-4 N2 Core Experiment on Automatic Segmentation Techniques, in which the authors are currently involved. Typical results obtained on MPEG4 sequences are here shown, in order to illustrate the segmentation algorithm performance.     

3D Shape Inferencing and Modeling for Video Retrieval

We present a geometry-based indexing approach for the retrieval of video databases. It consists of two modules: 3D object shape inferencing from video data and geometric modeling from the reconstructed shape structure. A motion-based segmentation algorithm employing feature block tracking and principal component split is used for multi-moving-object motion classification and segmentation. After segmentation, feature blocks from each individual object are used to reconstruct its motion and structure through a factorization method. The estimated shape structure and motion parameters are used to generate the implicit polynomial model for the object. The video data is retrieved using the geometric structure of objects and their spatial relationship. We generalize the 2D string to 3D to compactly encode the spatial relationship of objects.     

一种基于帧内图像分区的视频对象自动分割算法

视频运动对象的自动分割是实现新一代对象基视频编码标准MPEG-4的重要技术,本文提出了一种基于帧内图像分区的运动对象自动分割算法.首先以时域运动信息为依据利用高斯检验方法得到二值运动掩模图像,并建立MRF随机场模型进一步检验,然后提出了一种结合非线性变换的改进分水岭算法进行帧内图像分区,以划定对象区域与背景区域的界线,最后对时域分割和帧内分区结果进行比重运算,得到最终运动对象.针对MPEG-4标准测试序列和自采集手指视频序列的实验结果说明了算法的有效性.     

Unsupervised video object segmentation with distractor-aware online adaptation

Unsupervised video object segmentation is a crucial application in video analysis when there is no prior information about the objects. It becomes tremendously challenging when multiple objects occur and interact in a video clip. In this paper, a novel unsupervised video object segmentation approach via distractor-aware online adaptation (DOA) is proposed. DOA models spatiotemporal consistency in video sequences by capturing background dependencies from adjacent frames. Instance proposals are generated by the instance segmentation network for each frame and they are grouped by motion information as positives or hard negatives. To adopt high-quality hard negatives, the block matching algorithm is then applied to preceding frames to track the associated hard negatives. General negatives are also introduced when there are no hard negatives in the sequence. The experimental results demonstrate these two kinds of negatives are complementary. Finally, we conduct DOA using positive, negative, and hard negative masks to update the foreground and background segmentation. The proposed approach achieves state-of-the-art results on two benchmark datasets, the DAVIS 2016 and the Freiburg-Berkeley motion segmentation (FBMS)-59.     

Rank estimation in 3D multibody motion segmentation

A novel technique for rank estimation in 3D multibody motion segmentation is proposed. It is based on the study of the frequency spectra of moving rigid objects and does not use or assume a prior knowledge of the objects contained in the scene (i.e. number of objects and motion). The significance of rank estimation on multibody motion segmentation results is shown by using two motion segmentation algorithms over both synthetic and real data.     

2-D mesh-based video object segmentation and tracking with occlusion resolution

This paper integrates fully automatic video object segmentation and tracking including detection and assignment of uncovered regions in a 2-D mesh-based framework. Particular contributions of this work are (i) a novel video object segmentation method that is posed as a constrained maximum contrast path search problem along the edges of a 2-D triangular mesh, and (ii) a 2-D mesh-based uncovered region detection method along the object boundary as well as within the object. At the first frame, an optimal number of feature points are selected as nodes of a 2-D content-based mesh. These points are classified as moving (foreground) and stationary nodes based on multi-frame node motion analysis, yielding a coarse estimate of the foreground object boundary. Color differences across triangles near the coarse boundary are employed for a maximum contrast path search along the edges of the 2-D mesh to refine the boundary of the video object. Next, we propagate the refined boundary to the subsequent frame by using motion vectors of the node points to form the coarse boundary at the next frame. We detect occluded regions by using motion-compensated frame differences and range filtered edge maps. The boundaries of detected uncovered regions are then refined by using the search procedure. These regions are either appended to the foreground object or tracked as new objects. The segmentation procedure is re-initialized when unreliable motion vectors exceed a certain number. The proposed scheme is demonstrated on several video sequences.     

A novel moving object segmentation framework utilizing camera motion recognition for H.264 compressed videos

This paper presents a novel coarse to fine moving object segmentation framework for H.264/AVC compressed videos. The proposed framework integrates the global motion estimation and global motion compensation steps in the segmentation pipeline unlike previous techniques which did not consider such an integration. The integration is based on testing for presence of global motion by classifying the interframe motion vectors into moving camera class and still camera class. The decision boundary separating these two classes is learnt from the training video data. The integration automates the moving object segmentation to be applicable for static, moving and combination of static/moving camera cases which to the best of our knowledge has not been carried out earlier. Further, a novel coarse segmentation technique is proposed by decomposing the inter-frame motion vectors into wavelet sub-bands and utilizing logical operations on LH, HL and HH sub-band wavelet coefficients. The premise is based on the fact that since the LH, HL and HH sub-bands contain the detail information pertaining to horizontal, vertical and diagonal moving blocks respectively, they can be exploited to identify the coarse moving boundaries. The coarse segmentation is fast in comparison to state-of-the-art coarse segmentation methods as demonstrated by our experiments. Finally, these coarse boundaries are modeled in an energy minimization framework and shown that by minimizing the energy using graph cut optimization the segmentation is refined to obtain the fine segmentation. The proposed framework is tested on a number of standard video sequences encoded with H.264/AVC JM encoder and comparison is carried out with state-of-the-art compressed domain moving object segmentation methods as well as with an existing state-of-the-art pixel domain method to establish and validate the proposed moving object segmentation framework.     

Dynamic programming framework for automatic video object segmentation and vision-assisted video pre-processing

A generic definition of video objects, which is a group of pixels with temporal motion coherence, is considered. The generic video object (GVO) is the superset of the conventional video objects considered in the object segmentation literature. Because of its motion coherence, the GVO can be easily recognised by the human visual system. However, due to its arbitrary spatial distribution, the GVO cannot be easily detected by the existing algorithms which often assume the spatial homogeneousness of the video objects. The concept of extended optical flow is introduced and a dynamic programming framework for the GVO detection and segmentation is developed, whose solution is given by the Viterbi algorithm. Using this dynamic programming formulation, the proposed object detection algorithm is able to discover the motion path of the GVO automatically and refine its spatial region of support progressively. In addition to object segmentation, the proposed algorithm can also be applied to video pre-processing, removing the so-called 'video mask' noise in digital videos. Experimental results show that this type of vision-assisted video pre-processing significantly improves the compression efficiency.     

运动目标的自动分割与跟踪

该文提出了一种对视频序列中的运动目标进行自动分割的算法。该算法分析图像在L U V空间中的局部变化,同时使用运动信息来把目标从背景中分离出来。首先根据图像的局部变化,使用基于图论的方法把图像分割成不同的区域。然后,通过度量合成的全局运动与估计的局部运动之间的偏差来检测出运动的区域,运动的区域通过基于区域的仿射运动模型来跟踪到下一帧。为了提高提取的目标的时空连续性,使用Hausdorff跟踪器对目标的二值模型进行跟踪。对一些典型的MPEG-4测试序列所进行的评估显示了该算法的优良性能。