Computing occluding and transparent motions

Computing the motions of several moving objects in image sequences involves simultaneous motion analysis and segmentation. This task can become complicated when image motion changes significantly between frames, as with camera vibrations. Such vibrations make tracking in longer sequences harder, as temporal motion constancy cannot be assumed. The problem becomes even more difficult in the case of transparent motions.A method is presented for detecting and tracking occluding and transparent moving objects, which uses temporal integration without assuming motion constancy. Each new frame in the sequence is compared to a dynamic internal representation image of the tracked object. The internal representation image is constructed by temporally integrating frames after registration based on the motion computation. The temporal integration maintains sharpness of the tracked object, while blurring objects that have other motions. Comparing new frames to the internal representation image causes the motion analysis algorithm to continue tracking the same object in subsequent frames, and to improve the segmentation.     

Moving object detection and segmentation in urban environments from a moving platform

This paper proposes an effective approach to detect and segment moving objects from two time-consecutive stereo frames, which leverages the uncertainties in camera motion estimation and in disparity computation. First, the relative camera motion and its uncertainty are computed by tracking and matching sparse features in four images. Then, the motion likelihood at each pixel is estimated by taking into account the ego-motion uncertainty and disparity in computation procedure. Finally, the motion likelihood, color and depth cues are combined in the graph-cut framework for moving object segmentation. The efficiency of the proposed method is evaluated on the KITTI benchmarking datasets, and our experiments show that the proposed approach is robust against both global (camera motion) and local (optical flow) noise. Moreover, the approach is dense as it applies to all pixels in an image, and even partially occluded moving objects can be detected successfully. Without dedicated tracking strategy, our approach achieves high recall and comparable precision on the KITTI benchmarking sequences.     

Tracking human motion in structured environments using adistributed-camera system

This paper presents a comprehensive framework for tracking coarse human models from sequences of synchronized monocular grayscale images in multiple camera coordinates. It demonstrates the feasibility of an end-to-end person tracking system using a unique combination of motion analysis on 3D geometry in different camera coordinates and other existing techniques in motion detection, segmentation, and pattern recognition. The system starts with tracking from a single camera view. When the system predicts that the active camera will no longer have a good view of the subject of interest, tracking will be switched to another camera which provides a better view and requires the least switching to continue tracking. The nonrigidity of the human body is addressed by matching points of the middle line of the human image, spatially and temporally, using Bayesian classification schemes. Multivariate normal distributions are employed to model class-conditional densities of the features for tracking, such as location, intensity, and geometric features. Limited degrees of occlusion are tolerated within the system. Experimental results using a prototype system are presented and the performance of the algorithm is evaluated to demonstrate its feasibility for real time applications     

分割视频运动对象的研究

随着新的视频压缩标准MPEG-4的出现,如何从视频序列中分割出在语义上有意义的单独运动对象显得极其重要。文章从组成视频运动对象的分割系统出发,详细分析视频分割的各种方法如运动参数模型。变化检测掩模、图象分割及运动对象跟踪等,并对分割运动对象所采用的技术和方法进行了讨论。     

Prior-Free Dependent Motion Segmentation Using Helmholtz-Hodge Decomposition Based Object-Motion Oriented Map

Motion segmentation in moving camera videos is a very challenging task because of the motion dependence between the camera and moving objects. Camera motion compensation is recognized as an effective approach. However, existing work depends on prior-knowledge on the camera motion and scene structure for model selection. This is not always available in practice. Moreover, the image plane motion suffers from depth variations, which leads to depth-dependent motion segmentation in 3D scenes. To solve these problems, this paper develops a prior-free dependent motion segmentation algorithm by introducing a modified Helmholtz-Hodge decomposition (HHD) based object-motion oriented map (OOM). By decomposing the image motion (optical flow) into a curl-free and a divergence-free component, all kinds of camera-induced image motions can be represented by these two components in an invariant way. HHD identifies the camera-induced image motion as one segment irrespective of depth variations with the help of OOM. To segment object motions from the scene, we deploy a novel spatio-temporal constrained quadtree labeling. Extensive experimental results on benchmarks demonstrate that our method improves the performance of the state-of-the-art by 10%~20% even over challenging scenes with complex background.     

分裂合并运动分割的多运动视觉里程计方法

目的 视觉里程计（visual odometry,VO）仅需要普通相机即可实现精度可观的自主定位,已经成为计算机视觉和机器人领域的研究热点,但是当前研究及应用大多基于场景为静态的假设,即场景中只有相机运动这一个运动模型,无法处理多个运动模型,因此本文提出一种基于分裂合并运动分割的多运动视觉里程计方法,获得场景中除相机运动外多个运动目标的运动状态。方法 基于传统的视觉里程计框架,引入多模型拟合的方法分割出动态场景中的多个运动模型,采用RANSAC（random sample consensus）方法估计出多个运动模型的运动参数实例;接着将相机运动信息以及各个运动目标的运动信息转换到统一的坐标系中,获得相机的视觉里程计结果,以及场景中各个运动目标对应各个时刻的位姿信息;最后采用局部窗口光束法平差直接对相机的姿态以及计算出来的相机相对于各个运动目标的姿态进行校正,利用相机运动模型的内点和各个时刻获得的相机相对于运动目标的运动参数,对多个运动模型的轨迹进行优化。结果 本文所构建的连续帧运动分割方法能够达到较好的分割结果,具有较好的鲁棒性,连续帧的分割精度均能达到近100%,充分保证后续估计各个运动模型参数的准确性。本文方法不仅能够有效估计出相机的位姿,还能估计出场景中存在的显著移动目标的位姿,在各个分段路径中相机自定位与移动目标的定位结果位置平均误差均小于6%。结论 本文方法能够同时分割出动态场景中的相机自身运动模型和不同运动的动态物体运动模型,进而同时估计出相机和各个动态物体的绝对运动轨迹,构建出多运动视觉里程计过程。     

基于水平集的多运动目标时空分割与跟踪

针对背景运动时的运动目标分割问题,提出了一种对视频序列中的多个运动目标进行分割和跟踪的新方法。该方法着眼于运动的且较为复杂的背景,首先利用光流约束方程和背景运动模型建立一个基于时空域的能量函数,然后用该函数进行背景运动速度的估算和运动目标的分割和跟踪。而时空域中的运动目标的最佳分割,乃是通过使该能量函数最小化来驱动时空曲面演化实现。时空曲面的演化采用了水平集PDEs(Partial Differential Equations)方法。实验中,用实际的图像序列验证了该算法及其数值实现。实验表明,该方法能够同时进行背景运动速度的估算、运动目标的分割和跟踪。     

双摄像机系统对移动目标的跟踪

提出了一种由一个静止摄像机加上一个动态摄像机组成的双摄像机实时跟踪系统．该系统利用两种形式摄像机各自的优点，克服它们自身的不足，实现了对运动目标的实时跟踪．文中给出了双摄像机系统的组成以及功能划分．利用由近似的摄像机投影模型导出的单应性关系，实现两个摄像机图像之间的目标匹配．系统在静态摄像机的图像平面上建立目标的2D运动模型，采用卡尔曼滤波实现目标位置的预测，由单应性关系求出动态摄像机图像平面上对应目标的预测位置，然后计算摄像机动态平台的旋转角度，实现对动态平台的伺服控制．     

Human Motion Analysis: A Review

Human motion analysis is receiving increasing attention from computer vision researchers. This interest is motivated by a wide spectrum of applications, such as athletic performance analysis, surveillance, man–machine interfaces, content-based image storage and retrieval, and video conferencing. This paper gives an overview of the various tasks involved in motion analysis of the human body. We focus on three major areas related to interpreting human motion: (1) motion analysis involving human body parts, (2) tracking a moving human from a single view or multiple camera perspectives, and (3) recognizing human activities from image sequences. Motion analysis of human body parts involves the low-level segmentation of the human body into segments connected by joints and recovers the 3D structure of the human body using its 2D projections over a sequence of images. Tracking human motion from a single view or multiple perspectives focuses on higher-level processing, in which moving humans are tracked without identifying their body parts. After successfully matching the moving human image from one frame to another in an image sequence, understanding the human movements or activities comes naturally, which leads to our discussion of recognizing human activities.     

A framework for heading-guided recognition of human activity

A combined 2D, 3D approach is presented that allows for robust tracking of moving people and recognition of actions. It is assumed that the system observes multiple moving objects via a single, uncalibrated video camera. Low-level features are often insufficient for detection, segmentation, and tracking of non-rigid moving objects. Therefore, an improved mechanism is proposed that integrates low-level (image processing), mid-level (recursive 3D trajectory estimation), and high-level (action recognition) processes. A novel extended Kalman filter formulation is used in estimating the relative 3D motion trajectories up to a scale factor. The recursive estimation process provides a prediction and error measure that is exploited in higher-level stages of action recognition. Conversely, higher-level mechanisms provide feedback that allows the system to reliably segment and maintain the tracking of moving objects before, during, and after occlusion. Heading-guided recognition (HGR) is proposed as an efficient method for adaptive classification of activity. The HGR approach is demonstrated using “motion history images” that are then recognized via a mixture-of-Gaussians classifier. The system is tested in recognizing various dynamic human outdoor activities: running, walking, roller blading, and cycling. In addition, experiments with real and synthetic data sets are used to evaluate stability of the trajectory estimator with respect to noise.     

基于DSP的运动目标跟踪系统

描述了一种以TMS320C6701数字信号处理器为核心的高速图像处理板和图像实时采集卡及摄像头构成的实时运动跟踪系统。在对采集的实时图像序列进行如十预处理后．采用了金字塔结构的图像存储方式和特征点跟踪算法埘运动目标进行跟踪．通过对特征点的运算得到目标运动的偏差怍为摄像头运动的参数,是后根据这些参数控制摄像云台持续跟踪运动目标的移动,最后还给出了在复杂背景下跟踪人体的实验结果。     

Fast Electronic Digital Image Stabilization for Off-Road Navigation

Image stabilization can be used as front-end system for many tasks that require dynamic image analysis, such as navigation and tracking of independently moving objects from a moving platform. We present a fast and robust electronic digital image stabilization system that can handle large image displacements based on a two-dimensional feature-based multi-resolution motion estimation technique. The method tracks a small set of features and estimates the movement of the camera between consecutive frames. Stabilization is achieved by combining all motion from a reference frame and warping the current frame back to the reference. The system has been implemented on parallel pipeline image processing hardware (a Datacube MaxVideo 200) connected to a SUN SPARCstation 20/612 via a VME bus adaptor. Experimental results using video sequences taken from a camera mounted on a vehicle moving on rough terrain show the robustness of the system while running at approximately 20 frames/s.     

A computer vision system for the detection and classification of vehicles at urban road intersections

The paper presents a real-time vision system to compute traffic parameters by analyzing monocular image sequences coming from pole-mounted video cameras at urban crossroads. The system uses a combination of segmentation and motion information to localize and track moving objects on the road plane, utilizing a robust background updating, and a feature-based tracking method. It is able to describe the path of each detected vehicle, to estimate its speed and to classify it into seven categories. The classification task relies on a model-based matching technique refined by a feature-based one for distinguishing between classes having similar models, like bicycles and motorcycles. The system is flexible with respect to the intersection geometry and the camera position. Experimental results demonstrate robust, real-time vehicle detection, tracking and classification over several hours of videos taken under different illumination conditions. The system is presently under trial in Trento, a 100,000-people town in northern Italy.     

Automatic feature point extraction and tracking in image sequences for arbitrary camera motion

An automatic egomotion compensation based point correspondence algorithm is presented. A basic problem in autonomous navigation and motion estimation is automatically detecting and tracking features in consecutive frames, a challenging problem when camera motion is significant. In general, feature displacements between consecutive frames can be approximately decomposed into two components: (i) displacements due to camera motion which can be approximately compensated by image rotation, scaling, and translation; (ii) displacements due to object motion and/or perspective projection. In this paper, we introduce a two-step approach: First, the motion of the camera is compensated using a computational vision based image registration algorithm. Then consecutive frames are transformed to the same coordinate system and the feature correspondence problem is solved as though tracking moving objects for a stationary camera. Methods of subpixel accuracy feature matching, tracking and error analysis are introduced. The approach results in a robust and efficient algorithm. Results on several real image sequences are presented.The support of the Advanced Research Projects Agency (ARPA Order No. 8459) and the U.S. Army Engineer Topographic Laboratories under Contract DACA 76-92-C-0009 is gratefully acknowledged.     

一种时空联合的视频运动目标提取与跟踪新算法

提出了一种时空联合的视频运动目标提取与跟踪新算法。在空域分割中,针对分水岭方法过分割现象明显的缺点,对分水岭分割方法进行了改进;在时域分割中,首先对全局运动进行了补偿,随后为消除仅用两帧帧差进行对象分割所带来的误差,采用多帧帧差求和的方法,并自适应选取累积帧差的二值化阈值;时空分割结果进行投影融合后得到视频对象,提出用一种基于区域子块匹配的方法跟踪视频对象。实验结果表明,该算法简洁有效,能较好地把对象从运动背景中提取出来,并实现跟踪。     

Integrating tracking with fine object segmentation

We present a novel method for on-line, joint object tracking and segmentation in a monocular video captured by a possibly moving camera. Our goal is to integrate tracking and fine segmentation of a single, previously unseen, potentially non-rigid object of unconstrained appearance, given its segmentation in the first frame of an image sequence as the only prior information. To this end, we tightly couple an existing kernel-based object tracking method with Random Walker-based image segmentation. Bayesian inference mediates between tracking and segmentation, enabling effective data fusion of pixel-wise spatial and color visual cues. The fine segmentation of an object at a certain frame provides tracking with reliable initialization for the next frame, closing the loop between the two building blocks of the proposed framework. The effectiveness of the proposed methodology is evaluated experimentally by comparing it to a large collection of state of the art tracking and video-based object segmentation methods on the basis of a data set consisting of several challenging image sequences for which ground truth data is available.     

基于帧差和小波包分析算法的运动目标检测

提出了一种在镜头不动的情况下基于累积帧差分割和小波包分析融合技术的运动目标检测方法.这种方法可分为四步:使用改进的累积帧差算法和阈值分割算法完成目标区域的分割,并获得初始运动模板;利用小波包分析算法提取出单帧图像的边缘信息并获得细化的目标区域边缘图;根据初始运动模板和空域边缘图像的融合得到更精确的运动目标模板;最后结合原序列图像检测出完整的运动目标.实验结果表明:这种方法可以有效地从对比度较小和噪声较大的视频序列中较精确地检测出完整的运动目标.     

基于帧差和小波包分析算法的运动目标检测

提出了一种在镜头不动的情况下基于累积帧差分割和小波包分析融合技术的运动目标检测方法。这种方法可分为四步:使用改进的累积帧差算法和阈值分割算法完成目标区域的分割,并获得初始运动模板;利用小波包分析算法提取出单帧图像的边缘信息并获得细化的目标区域边缘图;根据初始运动模板和空域边缘图像的融合得到更精确的运动目标模板;最后结合原序列图像检测出完整的运动目标。实验结果表明:这种方法可以有效地从对比度较小和噪声较大的视频序列中较精确地检测出完整的运动目标。     

A General Framework for the Selection of World Coordinate Systems in Perspective and Catadioptric Imaging Applications

An imaging system with a single effective viewpoint is called a central projection system. The conventional perspective camera is an example of central projection system. A catadioptric realization of omnidirectional vision combines reflective surfaces with lenses. Catadioptric systems with an unique projection center are also examples of central projection systems. Whenever an image is acquired, points in 3D space are mapped into points in the 2D image plane. The image formation process represents a transformation from ³ to ², and mathematical models can be used to describe it. This paper discusses the definition of world coordinate systems that simplify the modeling of general central projection imaging. We show that an adequate choice of the world coordinate reference system can be highly advantageous. Such a choice does not imply that new information will be available in the images. Instead the geometric transformations will be represented in a common and more compact framework, while simultaneously enabling newer insights. The first part of the paper focuses on static imaging systems that include both perspective cameras and catadioptric systems. A systematic approach to select the world reference frame is presented. In particular we derive coordinate systems that satisfy two differential constraints (the compactness and the decoupling constraints). These coordinate systems have several advantages for the representation of the transformations between the 3D world and the image plane. The second part of the paper applies the derived mathematical framework to active tracking of moving targets. In applications of visual control of motion the relationship between motion in the scene and image motion must be established. In the case of active tracking of moving targets these relationships become more complex due to camera motion. Suitable world coordinate reference systems are defined for three distinct situations: perspective camera with planar translation motion, perspective camera with pan and tilt rotation motion, and catadioptric imaging system rotating around an axis going through the effective viewpoint and the camera center. Position and velocity equations relating image motion, camera motion and target 3D motion are derived and discussed. Control laws to perform active tracking of moving targets using visual information are established.