Segmentation-based tracking by support fusion

In this paper we present a novel fusion framework to combine the diverse outputs of arbitrary trackers, which are typically not directly combinable, allowing for significantly increasing the tracking quality. Our main idea is first to transform individual tracking outputs such as motion inliers, bounding boxes, or specific target image features to a shared pixel-based representation and then to run a fusion step on this representation. The fusion process additionally provides a segmentation, which, in turn, further allows for a dynamic weighting of the specific trackers’ contributions. In particular, we demonstrate our fusion concept by combining three diverse heterogeneous tracking approaches that significantly differ in methodology as well as in their reported outputs. In the experiments we show that the proposed fusion strategy can successfully handle highly complex non-rigid object scenarios where the individual trackers and state-of-the-art (non-rigid object and fusion based) trackers fail. We demonstrate high performance on a large number of challenging sequences, where we clearly outperform the individual trackers as well as state-of-the-art tracking approaches.     

融合特征的卫星视频车辆单目标跟踪

目的 卫星视频作为新兴遥感数据,可以提供观测区域高分辨率的空间细节信息与丰富的时序变化信息,为交通监测与特定车辆目标跟踪等应用提供了不同于传统视频视角的信息。相较于传统视频数据,卫星视频中的车辆目标分辨率低、尺度小、包含的信息有限。因此,当目标边界不明、存在部分遮挡或者周边环境表观模糊时,现有的目标跟踪器往往存在严重的目标丢失问题。对此,本文提出一种基于特征融合的卫星视频车辆核相关跟踪方法。方法 对车辆目标使用原始像素和方向梯度直方图（histogram of oriented gradient,HOG）方法提取包含互补判别能力的特征,利用核相关目标跟踪器分别得到具备不变性和判别性的响应图;通过响应图融合的方式结合两种特征的互补信息,得到目标位置;使用响应分布指标（response distribution criterion,RDC）判断当前目标特征的稳定性,决定是否更新跟踪器的表征模型。本文使用的相关滤波方法具有计算量小且运算速度快的特点,具备跟踪多个车辆目标的拓展能力。结果 在8个卫星视频序列上与主流的6种相关滤波跟踪器进行比较,实验数据涵盖光照变化、快速转弯、部分遮挡、阴影干扰、道路颜色变化和相似目标临近等情况,使用准确率曲线和成功率曲线的曲线下面积（area under curve,AUC）对车辆跟踪的精度进行评价。结果表明,本文方法较好地均衡了使用不同特征的基础跟踪器（性能排名第2）的判别能力,准确率曲线AUC提高了2.9%,成功率曲线AUC下降了4.1%,成功跟踪车辆目标,不发生丢失,证明了本文方法的先进性和有效性。结论 本文提出的特征融合的卫星视频车辆核相关跟踪方法,均衡了不同特征提取器的互补信息,较好解决了卫星视频中车辆目标信息不足导致的目标丢失问题,提升了精度。     

Robust visual tracking using information theoretical learning

This paper presents a novel online object tracking algorithm with sparse representation for learning effective appearance models under a particle filtering framework. Compared with the state-of-the-art ? ₁ sparse tracker, which simply assumes that the image pixels are corrupted by independent Gaussian noise, our proposed method is based on information theoretical Learning and is much less sensitive to corruptions; it achieves this by assigning small weights to occluded pixels and outliers. The most appealing aspect of this approach is that it can yield robust estimations without using the trivial templates adopted by the previous sparse tracker. By using a weighted linear least squares with non-negativity constraints at each iteration, a sparse representation of the target candidate is learned; to further improve the tracking performance, target templates are dynamically updated to capture appearance changes. In our template update mechanism, the similarity between the templates and the target candidates is measured by the earth movers’ distance(EMD). Using the largest open benchmark for visual tracking, we empirically compare two ensemble methods constructed from six state-of-the-art trackers, against the individual trackers. The proposed tracking algorithm runs in real-time, and using challenging sequences performs favorably in terms of efficiency, accuracy and robustness against state-of-the-art algorithms.     

基于活动基模型的非刚体目标跟踪算法研究

近年来,非刚体目标跟踪技术作为视频目标跟踪中的一个难点受到了广泛关注。为了精确跟踪非刚体目标,克服跟踪过程中目标形状变化和遮挡带来的困难,提出一种基于活动基模型的非刚体目标跟踪算法。首先采用共享草图算法从目标训练样本集中学习得到目标的活动基模型,然后把活动基模型嵌入粒子滤波观测模型中。在对金鱼与企鹅序列跟踪的实验结果表明,与现有算法相比,该算法在非刚体目标形状变化以及存在遮挡的情况下,具有更好的跟踪性能。     

变分调整约束下的反向低秩稀疏学习目标跟踪

目的 低秩稀疏学习目标跟踪算法在目标快速运动和严重遮挡等情况下容易出现跟踪漂移现象,为此提出一种变分调整约束下的反向低秩稀疏学习目标跟踪算法。方法 采用核范数凸近似低秩约束描述候选粒子间的时域相关性,去除不相关粒子,适应目标外观变化。通过反向稀疏表示描述目标表观,用候选粒子稀疏表示目标模板,减少在线跟踪中L₁优化问题的数目,提高跟踪效率。在有界变差空间利用变分调整对稀疏系数差分建模,约束目标表观在相邻帧间具有较小变化,但允许连续帧间差异存在跳跃不连续性,以适应目标快速运动。结果 实验利用OTB（object tracking benchmark）数据集中的4组涵盖了严重遮挡、快速运动、光照和尺度变化等挑战因素的标准视频序列进行测试,定性和定量对比了本文算法与5种热点算法的跟踪效果。定性分析基于视频序列的主要挑战因素进行比较,定量分析通过中心点位置误差（central pixel error,CPE）比较跟踪算法的精度。与CNT（convolutional networks training）、SCM（sparse collaborative model）、IST（inverse sparse tracker）、DDL（discriminative dictionary learning）和LLR（locally low-rank representation）算法相比,平均CPE值分别提高了2.80、4.16、13.37、35.94和41.59。实验结果表明,本文算法达到了较高的跟踪精度,对上述挑战因素更具鲁棒性。结论 本文提出的跟踪算法,综合了低秩稀疏学习和变分优化调整的优势,在复杂场景下具有较高的跟踪精度,特别是对严重遮挡和快速运动情况的有效跟踪更具鲁棒性。     

Online learning neural tracker

Object tracking is a fundamental computer vision problem and is required for many high-level tasks such as activity recognition, behavior analysis and surveillance. The main challenge in the object tracking problem is the dynamic change in object/background appearance, illumination, shape and occlusion. We present an online learning neural tracker (OLNT) to differentiate the object from the background and also adapt to changes in object/background dynamics. For target modeling and object tracking, a neural algorithm based on risk sensitive loss function is proposed to handle issues related to sample imbalance and dynamics of object. Region-based features like region-based color moments for larger mobile objects and color/texture features at pixel level for smaller mobile objects are used to discriminate the object from background. The proposed neural classifier automatically determines the number of neurons required to estimate the posterior probability map. In the online learning neural classifier, only one neuron parameter is updated per tracker to reduce the computational burden during online adaptation. The tracked object is represented using an estimated posterior probability map. The posterior probability map is used to adapt the bounding box to handle the scale change and improper initialization.For illustrating the advantage of the proposed OLNT under rapid illumination variation, change in appearance, scale/size change, and occlusion, we present results from benchmark video sequences. Finally, we also present the comparison with well-known trackers in the literature and highlight the advantage of the proposed tracker.     

基于区域相关性的全局运动视频中非刚性对象跟踪

提出一种适合全局运动视频中自动探测与跟踪非刚性对象的OT-GAV模型.该模型首先利用基于区域相关性的RDM算法计算相邻帧区域匹配,并结合Q学习与K-S统计法优化匹配结果,获得较为精确的区域运动向量.然后,利用前景和背景存在的运动形态差异,区域动态纹理一致性及对象运动过程中保持区域完整性的特点,逐步实现前景对象区域的探测与合并.实验证明,本模型及其相关算法可在室内和室外环境下,自动探测前景关注对象,获得其较为精确的边缘信息,并实施有效的跟踪.同时,该模型还能够解决对象跟踪过程中的"空洞"问题.     

Multi-object tracking evaluated on sparse events

This article presents a visual object tracking method and applies an event-based performance evaluation metric for assessment. The proposed monocular object tracker is able to detect and track multiple object classes in non-controlled environments. The tracking framework uses Bayesian per-pixel classification to segment an image into foreground and background objects, based on observations of object appearances and motions in real-time. Furthermore, a performance evaluation method is presented and applied to different state-of-the-art trackers based on successful detections of semantically high level events. These events are extracted automatically from the different trackers an their varying types of low level tracking results. Then, a general new event metric is used to compare our tracking method with the other tracking methods against ground truth of multiple public datasets.     

Structural similarity-based object tracking in multimodality surveillance videos

This paper addresses the problem of object tracking in video sequences for surveillance applications by using a recently proposed structural similarity-based image distance measure. Multimodality surveillance videos pose specific challenges to tracking algorithms, due to, for example, low or variable light conditions and the presence of spurious or camouflaged objects. These factors often cause undesired luminance and contrast variations in videos produced by infrared sensors (due to varying thermal conditions) and visible sensors (e.g., the object entering shadowy areas). Commonly used colour and edge histogram-based trackers often fail in such conditions. In contrast, the structural similarity measure reflects the distance between two video frames by jointly comparing their luminance, contrast and spatial characteristics and is sensitive to relative rather than absolute changes in the video frame. In this work, we show that the performance of a particle filter tracker is improved significantly when the structural similarity-based distance is applied instead of the conventional Bhattacharyya histogram-based distance. Extensive evaluation of the proposed algorithm is presented together with comparisons with colour, edge and mean-shift trackers using real-world surveillance video sequences from multimodal (infrared and visible) cameras.     

Visual tracking with conditionally adaptive multiple template update scheme for intricate videos

Tracking of moving objects in real-time scenes is a challenging research problem in computer vision. This is due to incessant live changes in the object features, background, occlusions, and illumination deviations occurring at different instances in the scene. With the objective of tracking visual objects in intricate videos, this paper presents a new color-independent tracking approach, the contributions of which are threefold. First, the illumination level of the sequences is maintained constant using fast discrete curvelet transform. Fisher information metric is calculated based on a cumulative score by comparing the template patches with a reference template at different timeframes. This metric is used for quantifying distances between the consecutive frame histogram distributions. Then, a novel iterative algorithm called conditionally adaptive multiple template update is proposed to regulate the object templates for handling dynamic occlusions effectively. The proposed method is evaluated on a set of extensive challenging benchmark datasets. Experimental results in terms of Center Location Error (CLE), Tracking Success Score (TSS), and Occlusion Success Score (OSS) show that the proposed method competes well with other relevant state-of-the-art tracking methods.     

基于模糊逻辑的多相机非重叠场景的物体匹配

An approach based on fuzzy logic for matching both articulated and non-articulated objects across multiple non-overlapping field of views (FoVs) from multiple cameras is proposed. We call it fuzzy logic matching algorithm (FLMA). The approach uses the information of object motion, shape and camera topology for matching objects across camera views. The motion and shape information of targets are obtained by tracking them using a combination of ConDensation and CAMShift tracking algorithms. The information of camera topology is obtained and used by calculating the projective transformation of each view with the common ground plane. The algorithm is suitable for tracking non-rigid objects with both linear and non-linear motion. We show videos of tracking objects across multiple cameras based on FLMA. From our experiments, the system is able to correctly match the targets across views with a high accuracy.     

Lazy texture selection based on active learning

Interactive selection of desired textures and textured objects from a video is a challenging problem in video editing. In this paper, we present a scalable framework that accurately selects textured objects with only moderate user interaction. Our method applies the active learning methodology, and the user only needs to label minimal initial training data and subsequent query data. An active learning algorithm uses these labeled data to obtain an initial classifier and iteratively improves it until its performance becomes satisfactory. A revised graph-cut algorithm based on the trained classifier has also been developed to improve the spatial coherence of selected texture regions. We show that our system is responsive even with videos of a large number of frames, and it frees the user from extensive labeling work. A variety of operations, such as color editing, compositing, and texture cloning, can be then applied to the selected textures to achieve interesting editing effects.     

Articulated body motion tracking using illumination invariant optical flow

We propose a model-based tracking method for articulated objects in monocular video sequences under varying illumination conditions. The tracking method uses estimates of optical flows constructed by projecting model textures into the camera images and comparing the projected textures with the recorded information. An articulated body is modelled in terms of 3D primitives, each possessing a specified texture on its surface. An important step in model-based tracking of 3D objects is the estimation of the pose of the object during the tracking process. The optimal pose is estimated by minimizing errors between the computed optical flow and the projected 2D velocities of the model textures. This estimation uses a least-squares method with kinematic constraints for the articulated object and a perspective camera model. We test our framework with an articulated robot and show results.     

均衡化特征匹配的非刚体细胞形态跟踪

提出利用均衡化特征匹配来进行非刚性细胞形体跟踪的方法。采用重启动的随机游走方法建立并求解特征匹配概率模型,利用双向均衡方法对匹配邻接矩阵进行均衡化处理,得到指定目标与待跟踪目标之间的精确匹配,以获得目标的定位跟踪结果。同时利用特征匹配结果进行目标的自动标定,并应用图像分割方法进行目标的精确轮廓跟踪。实验结果表明,将该方法应用于视频中动态背景下的运动细胞形态跟踪时,在背景相似度较高及目标迅速移动的条件下,表现出了良好的性能,与同类方法相比可获得较高的定位精度以及更为准确的目标轮廓。     

Fuzzy spatial constraints and ranked partitioned sampling approach for multiple object tracking

While particle filters are now widely used for object tracking in videos, the case of multiple object tracking still raises a number of issues. Among them, a first, and very important, problem concerns the exponential increase of the number of particles with the number of objects to be tracked, that can make some practical applications intractable. To achieve good tracking performances, we propose to use a Partitioned Sampling method in the estimation process with an additional feature about the ordering sequence in which the objects are processed. We call it Ranked Partitioned Sampling, where the optimal order in which objects should be processed and tracked is estimated jointly with the object state. Another essential point concerns the modeling of possible interactions between objects. As another contribution, we propose to represent these interactions within a formal framework relying on fuzzy sets theory. This allows us to easily model spatial constraints between objects, in a general and formal way. The association of these two contributions was tested on typical videos exhibiting difficult situations such as partial or total occlusions, and appearance or disappearance of objects. We show the benefit of using conjointly these two contributions, in comparison to classical approaches, through multiple object tracking and articulated object tracking experiments on real video sequences. The results show that our approach provides less tracking errors than those obtained with the classical Partitioned Sampling method, without the need for increasing the number of particles.     

Motion imitation with a handheld camera

In this paper, we present a novel method to extract motion of a dynamic object from a video that is captured by a handheld camera, and apply it to a 3D character. Unlike the motion capture techniques, neither special sensors/trackers nor a controllable environment is required. Our system significantly automates motion imitation which is traditionally conducted by professional animators via manual keyframing. Given the input video sequence, we track the dynamic reference object to obtain trajectories of both 2D and 3D tracking points. With them as constraints, we then transfer the motion to the target 3D character by solving an optimization problem to maintain the motion gradients. We also provide a user-friendly editing environment for users to fine tune the motion details. As casual videos can be used, our system, therefore, greatly increases the supply source of motion data. Examples of imitating various types of animal motion are shown.     

Medical image segmentation, volume representation and registration using spheres in the geometric algebra framework

This paper presents an algorithm to model volumetric data and other one for non-rigid registration of such models using spheres formulated in the geometric algebra framework. The proposed algorithm for modeling, as opposite to the Union of Spheres method, reduces the number of entities (spheres) used to model 3D data. Our proposal is based in marching cubes idea using, however, spheres, while the Union of Spheres uses Delaunay tetrahedrization. The non-rigid registration is accomplished in a deterministic annealing scheme. At the preprocessing stage we segment the objects of interest by a segmentation method based on texture information. This method is embedded in a region growing scheme. As our final application, we present a scheme for surgical object tracking using again geometric algebra techniques.