一个有效的基于运动的视频检索系统

在视频摄取的过程中,摄象机的运动在某种程度上反映了当前视频的部分语义特征,所以通过对摄象机运动的提取能够更好地实现对视频的浏览以及基于内容的检索。针对以上应用,本文提出了一种用运动矢量的分布来提取摄相机运动的方法,测试后表明,该方法能有效地分析出多种相机运动并具有较好的鲁棒性。同时我们以此为基础实现了一个基于相机运动的视频检索系统。     

基于FOE和改进MCMC的视频运动目标跟踪方法

改进的马尔可夫链蒙特卡洛MCMC(Markov Chain Monte Carlo)粒子滤波跟踪算法可以实现稳定跟踪多目标的目的。但在运动场景下,常常出现跟丢或者误跟的情况。考虑到相机聚焦中心FOE(Focus Of Expansion)在估计摄像头运动方面有不可替代的作用,首先通过构建FOE与目标在视频中位置的一个简单估计模型,估计目标的位置,再通过FOE与MCMC的结合,改善了目标丢失和抖动的现象,达到更加准确估计目标的目的。实验表明该方法对摄像头前后平移运动有比较理想的效果。     

Camera Motion Estimation Through Planar Deformation Determination

In this paper, we propose a global method for estimating the motion of a camera which films a static scene. Our approach is direct, fast and robust, and deals with adjacent frames of a sequence. It is based on a quadratic approximation of the deformation between two images, in the case of a scene with constant depth in the camera coordinate system. This condition is very restrictive but we show that, provided translation and depth inverse variations are small enough, the error on optical flow involved by the approximation of depths by a constant is small. In this context, we propose a new model of camera motion which allows to separate the image deformation in a similarity and a “purely” projective application, due to change of optical axis direction. This model leads to a quadratic approximation of image deformation that we estimate with an M-estimator; we can immediately deduce camera motion parameters. Electronic Supplementary Material  The online version of this article () contains supplementary material, which is available to authorized users.

Determining shape and motion from non-overlapping multi-camera rig: A direct approach using normal flows

In this paper, we explore how a wide field-of-view imaging system that consists of a number of cameras in a network arranged to approximate a spherical eye can reduce the complexity of estimating camera motion. Depth map of the imaged scene can be reconstructed once the camera motion is there. We present a direct method to recover camera motion from video data, which neither requires establishment of feature correspondences nor recovery of optical flow, but from normal flow which is directly observable. With a wide visual field, the inherent ambiguities between translation and rotation disappear. Several subsets of normal flow pairs and triplets can be utilized to constraint the directions of translation and rotation separately. The intersection of solution spaces arising from normal flow pairs or triplets yields the estimate on the direction of motion. In addition, the larger number of normal flow measurements so resulted can be used to combat the local flow extraction error. Rotational magnitude is recovered in a subsequent stage. This article details how motion recovery can be improved with the use of such an approximate spherical imaging system. Experimental results on synthetic and real image data are provided. The results show that the accuracy of motion estimation is comparable to those of the state-of-the-art methods that require to use explicit feature correspondences or full optical flows, and our method has a much faster computational speed.     

基于运动矢量的摄像机运动定性分类方法

在视频序列中,摄像机运动在某种程序上反映了当前视频的语分语义信息,知道了摄像机运动将能够取更好的实现对视频的游览以及检索,针对现有的依据光流分布来摄像机运动参数算法中的不足,给出了一种定性分析相机运动的方法,该方法能够在相机聚焦中心－ＦＯＥ（focus-of-expansion)不在成像平面中心时检测出给定类型的摄像机运动,实验结果表明该方法对于从视频序列中分析出给定类型的摄像机运动具有较的效果。     

Camera and light calibration from reflections on a sphere

This paper introduces a novel method for recovering light directions and camera parameters using a single sphere. Traditional methods for estimating light directions using spheres either assume both the radius and center of the sphere being known precisely, or they depend on multiple calibrated views to recover these parameters. In this paper, it will be shown that light directions can be uniquely determined from specular highlights observed in a single view of a sphere without knowing or recovering the exact radius and center of the sphere. Besides, given multiple views of the sphere, it will be shown that the focal length and the relative positions and orientations of the cameras can be determined using the recovered sphere and light directions. Closed form solutions for estimation of light directions and camera poses are presented, and an optimization procedure for estimation of the focal length is introduced. Experimental results on synthetic and real data demonstrates both the accuracy and robustness of the proposed method.     

基于压缩域行为描述子的视频检索新技术

构造了一个基于压缩域行为描述子的检索模型CMRS.在CMRS中,针对特征提取部分提出了一种基于宏块连接跟踪机制的运动轨迹提取算法和一种参数行为估计算法,并且对参数化运动原始的匹配准则（PSSM）加以改进,构造了新的基于速度的相似匹配准则.最后在检索模型的实现阶段,实现了模型的总体框架.     

Video synchronization and its application to object transfer

In video post-production applications, camera motion analysis and alignment are important in order to ensure the geometric correctness and temporal consistency. In this paper, we trade some generality in estimating and aligning camera motion for reduced computational complexity and increased image-based nature. The main contribution is to use fundamental ratios to synchronize video sequences of distinct scenes captured by cameras undergoing similar motions. We also present a simple method to align 3D camera trajectories when the fundamental ratios are not able to match the noisy trajectories. Experimental results show that our method can accurately synchronize sequences even when the scenes are totally different and have dense depths. An application on 3D object transfer is also demonstrated.     

专卖案件问询系统的设计和实现

介绍采用多媒体技术和网络技术组建专卖案件问询系统,该系统由问询室和指挥中心组成:问询室实现对专卖案件问询过程中的语音和图像进行采集和存储;指挥中心实现实时监控整个问询过程,领导和办案专家通过与问询室的交互,提示办案人员审讯的切入点,进行联合问询.     

A General Method for Sensor Planning in Multi-Sensor Systems: Extension to Random Occlusion

Systems utilizing multiple sensors are required in many domains. In this paper, we specifically concern ourselves with applications where dynamic objects appear randomly and the system is employed to obtain some user-specified characteristics of such objects. For such systems, we deal with the tasks of determining measures for evaluating their performance and of determining good sensor configurations that would maximize such measures for better system performance. We introduce a constraint in sensor planning that has not been addressed earlier: visibility in the presence of random occluding objects. occlusion causes random loss of object capture from certain necessitates the use of other sensors that have visibility of this object. Two techniques are developed to analyze such visibility constraints: a probabilistic approach to determine “average” visibility rates and a deterministic approach to address worst-case scenarios. Apart from this constraint, other important constraints to be considered include image resolution, field of view, capture orientation, and algorithmic constraints such as stereo matching and background appearance. Integration of such constraints is performed via the development of a probabilistic framework that allows one to reason about different occlusion events and integrates different multi-view capture and visibility constraints in a natural way. Integration of the thus obtained capture quality measure across the region of interest yields a measure for the effectiveness of a sensor configuration and maximization of such measure yields sensor configurations that are best suited for a given scenario. The approach can be customized for use in many multi-sensor applications and our contribution is especially significant for those that involve randomly occurring objects capable of occluding each other. These include security systems for surveillance in public places, industrial automation and traffic monitoring. Several examples illustrate such versatility by application of our approach to a diverse set of different and sometimes multiple system objectives. Most of this work was done while A. Mittal was with Real-Time Vision and Modeling Department, Siemens Corporate Research, Princeton, NJ 08540.     

Accurate Camera Calibration from Multi-View Stereo and Bundle Adjustment

The advent of high-resolution digital cameras and sophisticated multi-view stereo algorithms offers the promise of unprecedented geometric fidelity in image-based modeling tasks, but it also puts unprecedented demands on camera calibration to fulfill these promises. This paper presents a novel approach to camera calibration where top-down information from rough camera parameter estimates and the output of a multi-view-stereo system on scaled-down input images is used to effectively guide the search for additional image correspondences and significantly improve camera calibration parameters using a standard bundle adjustment algorithm (Lourakis and Argyros 2008). The proposed method has been tested on six real datasets including objects without salient features for which image correspondences cannot be found in a purely bottom-up fashion, and objects with high curvature and thin structures that are lost in visual hull construction even with small errors in camera parameters. Three different methods have been used to qualitatively assess the improvements of the camera parameters. The implementation of the proposed algorithm is publicly available at Furukawa and Ponce (2008b).     

一种用于视频监控的双摄像头系统实现

提出了一种新颖的用于视频监控的双摄像头系统,在此系统中全景摄像机与PTZ摄像机(云台摄像机)结合在一起,既能对大范围内的目标进行检测与跟踪又能对目标的详细图像进行捕捉。在全景摄像机获取的图像中进行运动检测,获取运动物体位置信息后利用PTZ摄像机对其进行检测分析,以实现二者的数据融合。设计了全景摄像机的反射镜面,对该双摄像头系统进行了标定,在实验室环境下的进行实验验证了系统的性能。     

A robust method to recognize critical configuration for camera calibration

When space points and camera optical center lie on a twisted cubic, no matter how many pairs there are used from the space points to their image points, camera parameters cannot be determined uniquely. This configuration is critical for camera calibration. We set up invariant relationship between six space points and their image points for the critical configuration. Then based on the relationship, an algorithm to recognize the critical configuration of at least six pairs of space and image points is proposed by using a constructed criterion function, where no any explicit computation on camera projective matrix or optical center is needed. Experiments show the efficiency of the proposed method.     

基于全景视觉的监控系统设计

设计了一个基于全景视觉的多摄像机监控网络。全景相机视野广,可以实现大范围的目标检测与跟踪。云台摄像机视角具有一定的自由度,可以捕捉目标的高分辨率图像。将全景相机与云台相机相互配合,通过多传感器的数据融合,分层次的跟踪算法及多相机调度算法,实现了大范围的多个运动目标的检测与跟踪,并能捕获目标的清晰图像。实验验证了该系统的有效性和合理性。     

A hierarchy of cameras for 3D photography

The view-independent visualization of 3D scenes is most often based on rendering accurate 3D models or utilizes image-based rendering techniques. To compute the 3D structure of a scene from a moving vision sensor or to use image-based rendering approaches, we need to be able to estimate the motion of the sensor from the recorded image information with high accuracy, a problem that has been well-studied. In this work, we investigate the relationship between camera design and our ability to perform accurate 3D photography, by examining the influence of camera design on the estimation of the motion and structure of a scene from video data. By relating the differential structure of the time varying plenoptic function to different known and new camera designs, we can establish a hierarchy of cameras based upon the stability and complexity of the computations necessary to estimate structure and motion. At the low end of this hierarchy is the standard planar pinhole camera for which the structure from motion problem is non-linear and ill-posed. At the high end is a camera, which we call the full field of view polydioptric camera, for which the motion estimation problem can be solved independently of the depth of the scene which leads to fast and robust algorithms for 3D Photography. In between are multiple view cameras with a large field of view which we have built, as well as omni-directional sensors.     

基于FPGA的高分辨率全景图像处理平台

以 FPGA 为核心,利用其内部的 Avalon 总线技术,在 SoPC Builder 中将 Flash 和 SRAM 从外设及自己设计的两个 DMA 采集和 DMA 显示主外设集成到一个 Avalon 系统中,构建出基于 FPGA的全景图像实时处理平台。结果证明,本系统可以完成对分辨率为2 048×2 048、15f/s 的 Camera Link接口全景图像的实时采集、存储并解算成适于观察的柱面图像,并以1024×768的分辨率实时显示。     

基于图像运动的三维重建及虚拟化应用

提出了一种稳定、快速地获取摄像机视频运动图像的三维重建方法,并对该运动图像做适当的虚拟化处理以展示重建效果。采用基于尺度不变特征点匹配的摄像机标定进行三维重建。尺度不变特征对于视频图像中的特征具有优秀敏锐的匹配能力,极大地放宽了摄像机标定对于设备上的限制,拓宽了实时三维重建的适用范围。通过对系统的一系列优化,不但提升了三维重建的精度,减少了错误匹配对摄像机标定的影响,而且进一步提升了处理速度。通过在三维重建的基础之上进行虚拟化处理,展示了本系统的三维重建效果。实验结果表明,该系统适用范围广,处理速度较快,重建精度高,实现了基于视频运动图像的三维重建。     

Self-calibration of wireless cameras with restricted degrees of freedom

This paper presents an approach for the automatic calibration of low-cost cameras which are assumed to be restricted in their freedom of movement to either pan or tilt movements. Camera parameters, including focal length, principal point, lens distortion parameter and the angle and axis of rotation, can be recovered from a minimum set of two images of the camera, provided that the axis of rotation between the two images goes through the camera’s optical center and is parallel to either the vertical (panning) or horizontal (tilting) axis of the image. Previous methods for auto-calibration of cameras based on pure rotations fail to work in these two degenerate cases. In addition, our approach includes a modified RANdom SAmple Consensus (RANSAC) algorithm, as well as improved integration of the radial distortion coefficient in the computation of inter-image homographies. We show that these modifications are able to increase the overall efficiency, reliability and accuracy of the homography computation and calibration procedure using both synthetic and real image sequences.     

An empirical evaluation of factors influencing camera calibration accuracy using three publicly available techniques

This paper presents an empirical study investigating the effects of training dataquantity, measurement error, pixel coordinate noise, and the choice of camera model, oncamera calibration accuracy, based on three publicly available techniques, developed byTsai, Heikkilä and Zhang. Results are first provided for asimulated camera system and then verified through carefully controlled experiments using real-world measurements. Our aims are to provide suggestions to researchers who require an immediate solution for camera calibration and a warning of the practical difficulties one may encounter in a real environment. In addition, we offer some insight into the factors to keep in mind when selecting a calibration technique. Finally, we hope that this paper can serve as an introduction to the field for those newly embarking upon calibration-related research. Wei Sun received the B.Eng. degree in Electronic Engineering from Shanghai Jiao Tong University in 1995 and the M.Sc. degree in Computer Science from Fudan University in 1998. At present, she is a Ph.D. candidate working at Centre for Intelligent Machines, Department of Eletrical and Computer Engineering of McGill University. Her research interests include image and video processing, computervision, machine learning and computer graphics. Jeremy R. Cooperstock (Ph.D. Toronto, 1996) is a professor of Electrical and Computer Engineering, a member of the Centre for Intelligent Machines, and a founding member of the Centre for Interdisciplinary Research in Music, Media and Technology at McGill University. Cooperstock is a member of the ACM and chairs the AES Technical Committee on Network Audio Systems. Cooperstock's research interests focus on computer mediation to facilitate high-fidelity human communication and the underlying technologies that support this goal.