Combining learning-based intensity distributions with nonparametric shape priors for image segmentation

Integration of shape prior information into level set formulations has led to great improvements in image segmentation in the presence of missing information, occlusion, and noise. However, most shape-based segmentation techniques incorporate image intensity through simplistic data terms. A common underlying assumption of such data terms is that the foreground and the background regions in the image are homogeneous, i.e., intensities are piecewise constant or piecewise smooth. This situation makes integration of shape priors inefficient in the presence of intensity inhomogeneities. In this paper, we propose a new approach for combining information from shape priors with that from image intensities. More specifically, our approach uses shape priors learned by nonparametric density estimation and incorporates image intensity distributions learned in a supervised manner. Such a combination has not been used in previous work. Sample image patches are used to learn the intensity distributions, and segmented training shapes are used to learn the shape priors. We present an active contour algorithm that takes these learned densities into account for image segmentation. Our experiments on synthetic and real images demonstrate the robustness of the proposed approach to complicated intensity distributions, and occlusions, as well as the improvements it provides over existing methods.     

A novel recursive bayesian learning-based method for the efficient and accurate segmentation of video with dynamic background

Segmentation of video with dynamic background is an important research topic in image analysis and computer vision domains. In this paper, we present a novel recursive Bayesian learning-based method for the efficient and accurate segmentation of video with dynamic background. In the algorithm, each frame pixel is represented as the layered normal distributions which correspond to different background contents in the scene. The layers are associated with a confident term and only the layers satisfy the given confidence which will be updated via the recursive Bayesian estimation. This makes learning of background motion trajectories more accurate and efficient. To improve the segmentation quality, the coarse foreground is obtained via simple background subtraction first. Then, a local texture correlation operator is introduced to fill the vacancies and remove the fractional false foreground regions. Extensive experiments on a variety of public video datasets and comparisons with some classical and recent algorithms are used to demonstrate its improvements in both segmentation accuracy and efficiency.     

Region-level motion-based background modeling and subtraction using MRFs.

This paper presents a new approach to automatic segmentation of foreground objects from an image sequence by integrating techniques of background subtraction and motion-based foreground segmentation. First, a region-based motion segmentation algorithm is proposed to obtain a set of motion-coherence regions and the correspondence among regions at different time instants. Next, we formulate the classification problem as a graph labeling over a region adjacency graph based on Markov random fields (MRFs) statistical framework. A background model representing the background scene is built and then is used to model a likelihood energy. Besides the background model, a temporal coherence is also maintained by modeling it as the prior energy. On the other hand, color distributions of two neighboring regions are taken into consideration to impose spatial coherence. Then, the a priori energy of MRFs takes both spatial and temporal coherence into account to maintain the continuity of our segmentation. Finally, a labeling is obtained by maximizing the a posteriori energy of the MRFs. Under such formulation, we integrate two different kinds of techniques in an elegant way to make the foreground detection more accurate. Experimental results for several video sequences are provided to demonstrate the effectiveness of the proposed approach.     

Unsupervised video object segmentation with distractor-aware online adaptation

Unsupervised video object segmentation is a crucial application in video analysis when there is no prior information about the objects. It becomes tremendously challenging when multiple objects occur and interact in a video clip. In this paper, a novel unsupervised video object segmentation approach via distractor-aware online adaptation (DOA) is proposed. DOA models spatiotemporal consistency in video sequences by capturing background dependencies from adjacent frames. Instance proposals are generated by the instance segmentation network for each frame and they are grouped by motion information as positives or hard negatives. To adopt high-quality hard negatives, the block matching algorithm is then applied to preceding frames to track the associated hard negatives. General negatives are also introduced when there are no hard negatives in the sequence. The experimental results demonstrate these two kinds of negatives are complementary. Finally, we conduct DOA using positive, negative, and hard negative masks to update the foreground and background segmentation. The proposed approach achieves state-of-the-art results on two benchmark datasets, the DAVIS 2016 and the Freiburg-Berkeley motion segmentation (FBMS)-59.     

Video object segmentation in rainy situations based on difference scheme with object structure and color analysis

A scheme based on a difference scheme using object structures and color analysis is proposed for video object segmentation in rainy situations. Since shadows and color reflections on the wet ground pose problems for conventional video object segmentation, the proposed method combines the background construction-based video object segmentation and the foreground extraction-based video object segmentation where pixels in both the foreground and background from a video sequence are separated using histogram-based change detection from which the background can be constructed and detection of the initial moving object masks based on a frame difference mask and a background subtraction mask can be further used to obtain coarse object regions. Shadow regions and color-reflection regions on the wet ground are removed from the initial moving object masks via a diamond window mask and color analysis of the moving object. Finally, the boundary of the moving object is refined using connected component labeling and morphological operations. Experimental results show that the proposed method performs well for video object segmentation in rainy situations.     

运动目标的自动分割与跟踪

该文提出了一种对视频序列中的运动目标进行自动分割的算法。该算法分析图像在L U V空间中的局部变化,同时使用运动信息来把目标从背景中分离出来。首先根据图像的局部变化,使用基于图论的方法把图像分割成不同的区域。然后,通过度量合成的全局运动与估计的局部运动之间的偏差来检测出运动的区域,运动的区域通过基于区域的仿射运动模型来跟踪到下一帧。为了提高提取的目标的时空连续性,使用Hausdorff跟踪器对目标的二值模型进行跟踪。对一些典型的MPEG-4测试序列所进行的评估显示了该算法的优良性能。     

基于空时多线索融合的超像素运动目标检测方法

运动目标检测是计算机视觉领域极具挑战性的难题,该文针对这一问题提出一种基于空时多线索融合的超像素运动目标检测方法。首先利用简单线性迭代聚类算法将当前帧分割为超像素集合,根据帧间的像素级时变线索找到当前帧中包含运动信息的前景超像素子块;然后根据运动目标的一致性原则建立前一帧目标模型,结合目标空间线索进一步确定包含运动目标的检测窗口,将目标检测问题转化为目标分割问题,利用密集角点检测将目标从窗口中分割出来。在多个具有挑战性的公开视频序列上同几种流行检测算法的实验对比结果证明了所提算法的有效性和优越性。     

Ultrasound image segmentation with shape priors: application to automatic cattle rib-eye area estimation.

Automatic ultrasound (US) image segmentation is a difficult task due to the quantity of noise present in the images and the lack of information in several zones produced by the acquisition conditions. In this paper, we propose a method that combines shape priors and image information to achieve this task. In particular, we introduce knowledge about the rib-eye shape using a set of images manually segmented by experts. A method is proposed for the automatic segmentation of new samples in which a closed curve is fitted taking into account both the US image information and the geodesic distance between the evolving curve and the estimated mean rib-eye shape in a shape space. This method can be used to solve similar problems that arise when dealing with US images in other fields. The method was successfully tested over a database composed of 610 US images, for which we have the manual segmentations of two experts.     

基于深度学习的复杂背景图像分类方法研究

复杂背景图像受背景干扰后不易被识别。针对这一问题,文中提出了基于前景分割机制的卷积神经网络图像分类方法。采用全卷积神经网络对图像前景区域进行自动分割,通过图像中前景区域周围的最小边界框对其进行定位。对于定位的前景区域,构建卷积神经网络对其进行处理以区分不同的类别,从而实现复杂背景图像的分类。将提出方法在公开数据集中提取的单一背景和复杂背景图像数据集上进行对比实验,并使用迁移学习与数据增强等方法优化模型。实验结果表明,所提方法使用前景区域分割相比于仅分类CNN具有更高的准确度,且复杂背景图像上的准确度提升幅度要远大于单一背景图像。该结果说明引入前景区域分割对于复杂背景图像分类模型准确度的提升具有一定帮助,能够显著前景区域特征并减少背景因素的干扰。     

Video inpainting under constrained camera motion.

A framework for inpainting missing parts of a video sequence recorded with a moving or stationary camera is presented in this work. The region to be inpainted is general: it may be still or moving, in the background or in the foreground, it may occlude one object and be occluded by some other object. The algorithm consists of a simple preprocessing stage and two steps of video inpainting. In the preprocessing stage, we roughly segment each frame into foreground and background. We use this segmentation to build three image mosaics that help to produce time consistent results and also improve the performance of the algorithm by reducing the search space. In the first video inpainting step, we reconstruct moving objects in the foreground that are "occluded" by the region to be inpainted. To this end, we fill the gap as much as possible by copying information from the moving foreground in other frames, using a priority-based scheme. In the second step, we inpaint the remaining hole with the background. To accomplish this, we first align the frames and directly copy when possible. The remaining pixels are filled in by extending spatial texture synthesis techniques to the spatiotemporal domain. The proposed framework has several advantages over state-of-the-art algorithms that deal with similar types of data and constraints. It permits some camera motion, is simple to implement, fast, does not require statistical models of background nor foreground, works well in the presence of rich and cluttered backgrounds, and the results show that there is no visible blurring or motion artifacts. A number of real examples taken with a consumer hand-held camera are shown supporting these findings.     

Medical image segmentation using minimal path deformable models with implicit shape priors.

This paper presents a new method for segmentation of medical images by extracting organ contours, using minimal path deformable models incorporated with statistical shape priors. In our approach, boundaries of structures are considered as minimal paths, i.e., paths associated with the minimal energy, on weighted graphs. Starting from the theory of minimal path deformable models, an intelligent "worm" algorithm is proposed for segmentation, which is used to evaluate the paths and finally find the minimal path. Prior shape knowledge is incorporated into the segmentation process to achieve more robust segmentation. The shape priors are implicitly represented and the estimated shapes of the structures can be conveniently obtained. The worm evolves under the joint influence of the image features, its internal energy, and the shape priors. The contour of the structure is then extracted as the worm trail. The proposed segmentation framework overcomes the short-comings of existing deformable models and has been successfully applied to segmenting various medical images.     

Automatic, robust global motion estimation using clustering

Global motion estimation (GME) is a vital part of many video compression and computer vision applications. However, the large moving foreground objects that are present in many video scenes make the task of GME more challenging. In this paper, we propose an automatic, efficient, and robust approach for GME that addresses the issue of large foreground objects. The proposed GME algorithm is based on two key ideas: a new clustering technique, to automate the initial segmentation of background and foreground blocks, and a modified Lorentzian estimator, to reduce the impact of any remaining foreground blocks on the GME process. We also apply an up-sampling technique to the estimated motion parameters to remove any errors caused by under-sampling during the warping process. These ideas provide a significant improvement in performance when combined into a common framework. Simulation results and analyses demonstrate the improved performance of our proposed algorithm over other state-of-the-art methods.     

动态背景下融合运动线索和颜色信息的视频目标分割算法

针对现有动态背景下目标分割算法存在的局限性,提出了一种融合运动线索和颜色信息的视频序列目标分割算法。首先,设计了一种新的运动轨迹分类方法,利用背景运动的低秩特性,结合累积确认的策略,可以获得准确的运动轨迹分类结果;然后,通过过分割算法获取视频序列的超像素集合,并计算超像素之间颜色信息的相似度;最后,以超像素为节点建立马尔可夫随机场模型,将运动轨迹分类信息以及超像素之间颜色信息统一建模在马尔可夫随机场的能量函数中,并通过能量函数最小化获得每个超像素的最优分类。在多组公开发布的视频序列中进行测试与对比,结果表明,本文方法可以准确分割出动态背景下的运动目标,并且较传统方法具有更高的分割准确率。     

背景渐变的视频对象分割算法研究及实现

提出了一种有效的背景渐变的视频对象分割算法.首先将前一帧分成前景和背景两部分,然后采用灰度投影匹配算法对当前帧进行全局运动估计和补偿,将当前帧与上一帧进行差分运算,便可得到差分图像.通过对差分图像进行二值化处理,得到运动模板并与前景信息进行相与计算,再结合当前帧信息便可得到运动目标.在TI公司的TMS320DM642芯片上验证了该算法,实验结果表明该算法不仅对亮度变化和环境变化具有鲁棒性,而且可独立、精确地分割出运动目标.     

Layered moving-object segmentation for stereoscopic video using motion and depth information

A novel layered stereoscopic moving-object segmentation method is proposed in this paper by exploiting both motion information and depth information to extract moving objects for each depth layer with high accuracy on their shape boundary. By taking a higher-order statistics on two frame-difference fields across three adjacent frames, the computed motion information are used to conduct change detection and generate one motion mask that consists of all the moving objects from all the depth layers involved at each view. It would be highly desirable, and challenging, to further differentiate them according to their residing depth layer to achieve layered segmentation. For that, multiple depth-layer masks are generated using our proposed disparity estimation method, one for each depth layer. By intersecting the motion mask and one depth-layer mask at any given layer-of-interest, the moving objects associated with the corresponding layer are then extracted. All the above-mentioned processes are repeatedly performed along the video sequence with a sliding window of three frames at a time. For demonstration, only the foreground and the background layers are considered in this paper, while the proposed method is generic and can be straightforwardly extended to more layers, once the corresponding depth-layer masks are made available. Experimental results have shown that the proposed layered moving-object segmentation method is able to segment the foreground and background moving objects separately, with high accuracy on their shape boundary. In addition, the required computational load is considered fairly inexpensive, since our design methodology is to generate masks and perform intersections for extracting the moving objects for each depth layer.     

基于运动矢量场的运动目标区域检测

基于稀疏运动矢量场,提出一种动态背景下的运动 目标区域检测方法。根据运动矢量场特性分析进行全局运动 参数估计和全局运动补偿,实现动态场景中的背景校正;利用最大树数据结构, 基于运动矢量补偿误差分级表示视频帧中 运动基本一致的连通区域,进行运动区域初始分类;根据运动目标在空间上的连通性和运动 一致性的特点,选择区域相似性 度量准则,进行区域合并和滤波,将具有相似运动的连通区域合并,实现运动目标区域检测 。将检测出的运动目标区域作为 运动矢量外点反过来又应用于全局运动参数估计过程中,全局运动估计和运动目标区域检测 交替地进行,不仅加快了它们的 计算速度,同时也提高了它们计算和检测的准确性。实验结果表明,本文算法能较好地补偿 序列的全局运动,有效地检测出 局部目标运动区域。     

基于运动矢量多级分析的视频全局运动估计

基于运动矢量场的视频全局运动估计相较于基于像素的估计方法具有较低的计算复杂度,因而广泛应用于视频分割及视频压缩等领域中。然而噪声和前景目标等外点区域的存在,降低了全局运动估计的准确性。为了提高全局运动估计的准确度,该文提出一种基于运动矢量多级分析的全局运动估计算法,该算法根据局部运动与全局运动的运动特性差异自适应地滤除前景目标区域,由邻域矢量间相似性度量检测出纹理平滑周期区域,最后滤除孤立的噪声区域,由滤波得到的内点区域求解全局运动参数。实验结果表明,该方法能有效地滤除外点区域,提高全局运动估计的准确性。     

新闻视频单元高效切分方法的研究与实现

提出了一个基于口播检测的高效新闻视频单元切分方法。该方法首先检测出新闻视频的镜头边界;然后从每个镜头中提取出关键帧,并计算出关键帧的直方图和SIFT特征;最后通过关键帧聚类获取新闻视频中的所有口播镜头,并以此为依据将新闻视频分割成多个语义单元。基于以上方法,开发了用于新闻视频单元切分的软件系统。该系统能够准确、高效地实现新闻单元的自动切分,有效地减轻视频切分时的工作强度,满足新媒体时代节目快速制作的要求。     

一种新的基于时空信息的视频分割

提出了一种新的基于时空信息的视频分割算法．即先将原始图像标记成不同的区域,然后以帧间差分得到的对象运动信息作为评判准则,将这些区域分别归类于前景对象和背景．达到对象分割的目的。特别是在区域标记的过程中,采用了一种新的基于分水岭的区域标识技术。通过对标准图像序列的实验结果可以看到,利用该算法能够较精确地分割出视频对象。