Composition and Retrieval of Visual Information for Video Databases

This paper presents a new visual aggregation model for representing visual information about moving objects in video data. Based on available automatic scene segmentation and object tracking algorithms, the proposed model provides eight operations to calculate object motions at various levels of semantic granularity. It represents trajectory, color and dimensions of a single moving object and the directional and topological relations among multiple objects over a time interval. Each representation of a motion can be normalized to improve computational cost and storage utilization. To facilitate query processing, there are two optimal approximate matching algorithms designed to match time-series visual features of moving objects. Experimental results indicate that the proposed algorithms outperform the conventional subsequence matching methods substantially in the similarity between the two trajectories. Finally, the visual aggregation model is integrated into a relational database system and a prototype content-based video retrieval system has been implemented as well.     

Efficient processing of probabilistic reverse nearest neighbor queries over uncertain data

Reverse nearest neighbor (RNN) search is very crucial in many real applications. In particular, given a database and a query object, an RNN query retrieves all the data objects in the database that have the query object as their nearest neighbors. Often, due to limitation of measurement devices, environmental disturbance, or characteristics of applications (for example, monitoring moving objects), data obtained from the real world are uncertain (imprecise). Therefore, previous approaches proposed for answering an RNN query over exact (precise) database cannot be directly applied to the uncertain scenario. In this paper, we re-define the RNN query in the context of uncertain databases, namely probabilistic reverse nearest neighbor (PRNN) query, which obtains data objects with probabilities of being RNNs greater than or equal to a user-specified threshold. Since the retrieval of a PRNN query requires accessing all the objects in the database, which is quite costly, we also propose an effective pruning method, called geometric pruning (GP), that significantly reduces the PRNN search space yet without introducing any false dismissals. Furthermore, we present an efficient PRNN query procedure that seamlessly integrates our pruning method. Extensive experiments have demonstrated the efficiency and effectiveness of our proposed GP-based PRNN query processing approach, under various experimental settings.     

DISCOV: A Framework for Discovering Objects in Video

This paper presents a probabilistic framework for discovering objects in video. The video can switch between different shots, the unknown objects can leave or enter the scene at multiple times, and the background can be cluttered. The framework consists of an appearance model and a motion model. The appearance model exploits the consistency of object parts in appearance across frames. We use maximally stable extremal regions as observations in the model and hence provide robustness to object variations in scale, lighting and viewpoint. The appearance model provides location and scale estimates of the unknown objects through a compact probabilistic representation. The compact representation contains knowledge of the scene at the object level, thus allowing us to augment it with motion information using a motion model. This framework can be applied to a wide range of different videos and object types, and provides a basis for higher level video content analysis tasks. We present applications of video object discovery to video content analysis problems such as video segmentation and threading, and demonstrate superior performance to methods that exploit global image statistics and frequent itemset data mining techniques.     

Multiresolution video object extraction fitted to scalable wavelet-based object coding

To enable content based functionalities in video processing algorithms, decomposition of scenes into semantic objects is necessary. A semi-automatic Markov random field based multiresolution algorithm is presented for video object extraction in a complex scene. In the first frame, spatial segmentation and user intervention determine objects of interest. The specified objects are subsequently tracked in successive frames and newly appeared objects/regions are also detected. The video object extraction algorithm includes discrete wavelet transform decomposition multiresolution Markov random field (MRF)-based spatial segmentation with emphasis on border smoothness at different resolutions, and an MRF-based backward region classification that determines the tracked objects in the scene. Finally, a motion constraint, embedded in the region classifier, determines the newly appeared objects/regions and completes the proposed algorithm towards an efficient video segmentation algorithm. The results are applicable for generic segmentation applications, however the proposed multiresolution video segmentation algorithm supports scalable object-based wavelet coding in particular. Moreover, compared to traditional object extraction algorithms, it produces smoother and more visually pleasing shape masks at different resolutions. The proposed effective multiresolution video object extraction method allows for larger motion, better noise tolerance and less computational complexity     

基于运动显著性的移动镜头下的运动目标检测

针对移动镜头下的运动目标检测中的背景建模复杂、计算量大等问题,提出一种基于运动显著性的移动镜头下的运动目标检测方法,在避免复杂的背景建模的同时实现准确的运动目标检测。该方法通过模拟人类视觉系统的注意机制,分析相机平动时场景中背景和前景的运动特点,计算视频场景的显著性,实现动态场景中运动目标检测。首先,采用光流法提取目标的运动特征,用二维高斯卷积方法抑制背景的运动纹理;然后采用直方图统计衡量运动特征的全局显著性,根据得到的运动显著图提取前景与背景的颜色信息;最后,结合贝叶斯方法对运动显著图进行处理,得到显著运动目标。通用数据库视频上的实验结果表明,所提方法能够在抑制背景运动噪声的同时,突出并准确地检测出场景中的运动目标。     

一种动态场景下基于时空信息的视频对象提取算法

在实际应用中,许多视频序列具有运动背景,使得从其中提取视频对象变得复杂,为此提出了一种基于运动估计和图形金字塔的动态场景下的视频对象提取算法。该算法首先引入了相位相关法求取运动向量,因避免了视频序列中光照变化的影响,故可提高效率和稳健性;接着再根据参数模型进行全局运动估计来得到最终运动模板;然后利用图形金字塔算法对当前模板内图像区域进行空间分割,最终提取出语义视频对象。与现有算法相比,对于从具有动态场景的视频流中提取运动对象的情况,由于使用该算法能有效地避开精准背景补偿,因而不仅节省了计算量,而且提取出来的语义对象精度较高。实验表明,无论是对动态场景中刚性还是非刚性运动物体的分割,该算法都具有较好的效果。     

Layered Representation of a Video Shot with Mosaicing

: This paper presents a motion segmentation method useful for representing efficiently a video shot as a static mosaic of the background plus sequences of the objects moving in the foreground. This generates an MPEG-4 compliant, layered representation useful for video coding, editing and indexing. First, a mosaic of the static background is computed by estimating the dominant motion of the scene. This is achieved by tracking features over the video sequence and using a robust technique that discards features attached to the moving objects. The moving objects get removed in the final mosaic by computing the median of the grey levels. Then, segmentation is obtained by taking the pixelwise difference between each frame of the original sequence and the mosaic of the background. To discriminate between the moving object and noise, temporal coherence is exploited by tracking the object in the binarised difference image sequence. The automatic computation of the mosaic and the segmentation procedure are illustrated with real sequences experiments. Examples of coding and content-based manipulation are also shown. Received: 31 August 2000, Received in revised form: 18 April 2001, Accepted: 20 July 2001     

Continuous visible k nearest neighbor query on moving objects

A visible k nearest neighbor (Vk NN) query retrieves k objects that are visible and nearest to the query object, where “visible” means that there is no obstacle between an object and the query object. Existing studies on the Vk NN query have focused on static data objects. In this paper we investigate how to process the query on moving objects continuously. We propose an effective filtering-and-refinement framework for evaluating this type of queries. We exploit spatial proximity and visibility properties between the query object and data objects to prune search space under this framework. A detailed cost analysis and a comprehensive experimental study are conducted on the proposed framework. The results validate the effectiveness of the pruning techniques and verify the efficiency of the proposed framework. The proposed framework outperforms a straightforward solution by an order of magnitude in terms of both communication and computation costs.     

异构数据源数据集成的研究

对象代理模型可以作为数据集成的一种通用数据模型。通过建立代理对象和源对象,查询处理的对应关系也能够较好地实现,应用可以将不同的、对于代理对象的查询处理翻译成对于局部数据源源对象的查询处理,也可以把对局部数据源的查询结果以用户应用想要的方式返回。对象代理模型可在Smalltalk环境中实现。本文讨论如何在Smalltalk
k环境中实现基于对象代理模型的异构信息源的集成。     

Efficiently instantiating view-objects from remote relational databases

View-objects are complex objects that are instantiated by delivering a query to a database and converting the query result into a nested structure. In relational databases, query results are conventionally retrieved as a single flat relation, which contains duplicate subtuples in its composite tuples. These duplicate subtuples increase the amount of data to be handled and thus degrade performance. In this article, we describe two new methods that retrieve a query result in structures other than a single flat relation. One method retrieves a set of relation fragments, and the other retrieves a single-nested relation. We first describe their algorithms and cost models, and then present the cost comparison results in a client-server architecture with a relational main memory database residing on a server.     

基于特征知识库的动漫视频暴力场景检测方法

针对海量动漫视频剪辑中可能包含不符合观众认知或接收程度的内容,为观众带来不愉快体验的问题,提出了基于知识库的卡通视频暴力场景检测方法。首先,利用低级特征(RBG颜色、亮度、色调)开发连续函数,进行场景中镜头边界识别;然后,利用运动信息,计算视频帧中的运动。接着,使用空间信息和运动信息进行分割处理,并使用欧氏距离方法进行对象识别;最后,利用包含所有常见物品、卡通角色及其视觉特征和行为特征的知识库,计算场景的暴力概率。实验结果表明,该系统能够成功检测到卡通视频中的暴力场景。     

Functional Query Optimization over Object-Oriented Views for Data Integration

AMOS is a mediator system that supports passive (non-intrusive) integration of data from heterogeneous and autonomous data sources. It is based on a functional data model and a declarative functional query language AMOSQL. Foreign data sources, e.g., relational databases, text files, or other types of data sources can be wrapped with AMOS mediators, making them accessible through AMOSQL. AMOS mediators can communicate among each other through the multi-database constructs of AMOSQL that allow definition of functional queries and OO views accessing other AMOS servers. The integrated views can contain both functions and types derived from the data sources. Furthermore, local data associated with these view definitions may be stored in the mediator database. This paper describes AMOS' multi-database query facilities and their optimization techniques. Calculus-based function transformations are used to generate minimal query expressions before the query decomposition and cost-based algebraic optimization steps take place. Object identifier (OID) generation is used for correctly representing derived objects in the mediators. A selective OID generation mechanism avoids overhead by generating in the mediator OIDs only for those derived objects that are either needed during the processing of a query or have associated local data in the mediator database. The validity of the derived objects that are assigned OIDs and the completeness of queries to the views are guaranteed by system generated predicates added to the queries.     

Fast and accurate global motion compensation

Video understanding has attracted significant research attention in recent years, motivated by interest in video surveillance, rich media retrieval and vision-based gesture interfaces. Typical methods focus on analyzing both the appearance and motion of objects in video. However, the apparent motion induced by a moving camera can dominate the observed motion, requiring sophisticated methods for compensating for camera motion without a priori knowledge of scene characteristics. This paper introduces two new methods for global motion compensation that are both significantly faster and more accurate than state of the art approaches. The first employs RANSAC to robustly estimate global scene motion even when the scene contains significant object motion. Unlike typical RANSAC-based motion estimation work, we apply RANSAC not to the motion of tracked features but rather to a number of segments of image projections. The key insight of the second method involves reliably classifying salient points into foreground and background, based upon the entropy of a motion inconsistency measure. Extensive experiments on established datasets demonstrate that the second approach is able to remove camera-based observed motion almost completely while still preserving foreground motion.     

室内场景视频结构化描述系统设计

视频结构化描述是对一种视频内容信息提取和应用的技术,它对视频内容按照语义关系,采用时空分割、特征提取、对象识别等处理手段,组织成可供计算机和人理解的文本信息的技术。本文介绍基于该技术的室内场景描述系统方案,系统实现对室内场景的描述以及相关描述数据的存储和分发。结果表明经过结构化描述的视频可提高应用效率。     

一种基于Kalman滤波的视频对象跟踪方法

为了更加准确地预测对象的位置和运动，利用刚体运动模型导出最佳Kalman系数，通过Kalman反馈滤波器对Moscheni等人提出的视频对象分割与跟踪算法进行改进，提出了一种将离散Kalman滤波技术用于视频序列的对象跟踪方法。这种方法可用于有关场景描述的各种应用领域中，如在机器视觉的研究中，对动态场景进行分析与理解；在基于对象的视频编码中（如MPEG-4），对视频对象进行分割后，分别进行编码，从而改善编码的可分级性及编码效率。实验结果表明，采用这种方法可以有效地改善时间-空间分割和目标跟踪，有助于更好地理解动态场景，并表现出良好的鲁棒性。     

Hierarchical database for a multi-camera surveillance system

This paper presents a framework for event detection and video content analysis for visual surveillance applications. The system is able to coordinate the tracking of objects between multiple camera views, which may be overlapping or non-overlapping. The key novelty of our approach is that we can automatically learn a semantic scene model for a surveillance region, and have defined data models to support the storage of tracking data with different layers of abstraction into a surveillance database. The surveillance database provides a mechanism to generate video content summaries of objects detected by the system across the entire surveillance region in terms of the semantic scene model. In addition, the surveillance database supports spatio-temporal queries, which can be applied for event detection and notification applications.     

文本对象查询的相关性计算

本文把特征文件，符号对象模型及时间戳排序等概念引入到文本对象查询系统的设计之中，提出了基于索引调组集的用户查询和文本对象相关性计算，以词组标识解决词组同义词等价性判定问题以及借助于时间戳排序技术充分利用查询反馈信息以提高系统时空效率的算法和方法，并讨论了面向文本数据库管理系统的文本对象查询的优化策略及逻辑实现等问题。     

立体视频对象分割及其三维重建算法研究*

为更加有效分析立体视频对象,本文提出了一种基于离散冗余小波变换的立体视频对象分割算法,首先采用离散冗余小波变换提取特征点结合DT网格技术的视差估计方法,获得了可靠的视差场,再利用视差信息对立体视频中静止对象进行分割。对于立体视频序列中的运动对象,采用离散冗余小波提取运动区域的方法进行分割。实验结果表明,本算法对有重叠的多视频对象具有较好的分割效果,可同时分割静止物体和运动物体,具有较好的精确性和鲁棒性。对于分割出的立体视频对象,结合深度信息对其进行三维重建,得到较好的三维效果。     

Design and Implementation of the ROL Deductive Object-Oriented Database System

ROL is a deductive object-oriented database system that has been implemented at the University of Regina. It provides a uniform rule-based declarative language for defining, manipulating and querying databases, which integrates important features of both deductive databases and object-oriented databases. It supports object identity, complex objects, classes, class hierarchies, multiple inheritance with overriding and blocking, and schema definition. It also supports structured values such as functor objects and sets, treating them as first class citizens, and providing powerful mechanisms for representing both partial and complete information about sets. This paper describes its design and implementation. An important novel feature of the implementation is that it combines top-down and bottom-up evaluation strategies and automatically selects a strategy based on the nature of the query and data in the database.