Region-based surveillance video retrieval with effective object representation

Retrieving the most relevant video frames that contain the object specified in a given query (query-by-region) remains a challenging task. Two common challenges of region-based retrieval approaches are to accurately extract or segment object(s) and select a proper matching strategy. This paper addresses these problems by proposing a retrieval approach that uses a new region-based matching technique equipped with an effective object representation method. In the first stage, the proposed approach selects the most informative instances of each object that appeared in the video by utilizing an adapted clustering algorithm over the extracted features. In the retrieval stage, the new matching technique returns the most relevant sequences of video by mapping a given region with those identified representative instances of objects based on their similarity scores. The proposed approach is evaluated on standard datasets and the results demonstrate a 31% improvement in the retrieval performance compared to other state-of-the-art methods.     

基于减法聚类算法的视频运动目标定位

针对视频运动目标定位的需要,本文给出了一种新的视频运动目标定位方法.该方法运用减法聚类算法对视频运动目标进行定位.分析了减法聚类算法的原理,给出了减法聚类算法的公式推导,目标定位的实现步骤及流程框图.研究了本文方法对不同类型视频运动目标的定位效果,并与基于区域生长的定位方法进行了详细比较.结合实验数据说明了本文方法的定位过程、处理时间及抗噪性能.实验结果表明,本文方法适用于待定位视频序列二值图像存在较大噪声斑点或空域连通特性较差的场合.     

An Algorithm for Mining Gradual Moving Object Clusters Pattern From Trajectory Streams

The discovery of gradual moving object clusters pattern from trajectory streams allows characterizing movement behavior in real time environment, which leverages new applications and services. Since the trajectory streams is rapidly evolving, continuously created and cannot be stored indefinitely in memory, the existing approaches designed on static trajectory datasets are not suitable for discovering gradual moving object clusters pattern from trajectory streams. This paper proposes a novel algorithm of gradual moving object clusters pattern discovery from trajectory streams using sliding window models. By processing the trajectory data in current window, the mining algorithm can capture the trend and evolution of moving object clusters pattern. Firstly, the density peaks clustering algorithm is exploited to identify clusters of different snapshots. The stable relationship between relatively few moving objects is used to improve the clustering efficiency. Then, by intersecting clusters from different snapshots, the gradual moving object clusters pattern is updated. The relationship of clusters between adjacent snapshots and the gradual property are utilized to accelerate updating process. Finally, experiment results on two real datasets demonstrate that our algorithm is effective and efficient.     

尺度方向自适应的减法聚类视频运动目标定位

针对减法聚类算法对视频运动目标进行定位时无法获取目标尺度及方向参数的问题,本文提出了一种可获取待定位目标尺度及方向参数的视频运动目标定位算法.该算法在减法聚类算法预定位目标位置及获得目标个数的基础上,进一步采用模糊C均值聚类对目标前景样本进行归类,最后通过对目标前景样本协方差矩阵特征值和特征向量的分析获得目标的尺度及方向参数,从而实现对视频运动目标的定位.实验结果表明,所提出的方法与原减法聚类定位方法相比可获得更合理的目标定位结果.     

Moving/motionless foreground object detection using fast statistical background updating

Abstract

In video surveillance, the detection of foreground objects in an image sequence from a still camera is very important for object tracking, activity recognition and behaviour understanding. The conventional background subtraction cannot respond promptly to dynamic changes in the background, and temporal difference cannot accurately extract the object shapes and detect motionless objects. In this paper, we propose a fast statistical process control scheme for foreground segmentation. The proposed method can promptly calculate the exact grey-level mean and standard deviation of individual pixels in both short- and long-term image sequences by simply deleting the earliest one among the set of images and adding the current image scene in the image sequence. A short-term updating process can be highly responsive to dynamic changes of the environment, and a long-term updating process can well extract the shape of a moving object. The detection results from both the short- and long-term processes are incorporated to detect motionless objects and eliminate non-stationary background objects. Experimental results have shown that the proposed scheme can be well applied to both indoor and outdoor environments. It can effectively extract foreground objects with various moving speeds or without motion at a high process frame rate.     

Visual Relationship Detection with Contextual Information

Understanding an image goes beyond recognizing and locating the objects in it, the relationships between objects also very important in image understanding. Most previous methods have focused on recognizing local predictions of the relationships. But real-world image relationships often determined by the surrounding objects and other contextual information. In this work, we employ this insight to propose a novel framework to deal with the problem of visual relationship detection. The core of the framework is a relationship inference network, which is a recurrent structure designed for combining the global contextual information of the object to infer the relationship of the image. Experimental results on Stanford VRD and Visual Genome demonstrate that the proposed method achieves a good performance both in efficiency and accuracy. Finally, we demonstrate the value of visual relationship on two computer vision tasks: image retrieval and scene graph generation.     

一种采用背景统计技术的视频对象分割算法

利用背景统计技术从累积的帧差信息中构建出完整、可靠的背景区域,并将其与当前帧相比较,得到初始对象分割掩膜;再对之进行后处理,以消除噪声影响和平滑对象边界,从而获得较好的对象分割掩膜,并提取出视频对象。该算法不需要预知运动对象的形状、数目等,就能较好地从静止背景中分离出目标,实验证明,它具有一定的实用性和鲁棒性。     

Moving object detection and tracking in videos through turbulent medium

This paper addresses the problem of identifying and tracking moving objects in a video sequence having a time-varying background. This is a fundamental task in many computer vision applications, though a very challenging one because of turbulence that causes blurring and spatiotemporal movements of the background images. Our proposed approach involves two major steps. First, a moving object detection algorithm that deals with the detection of real motions by separating the turbulence-induced motions using a two-level thresholding technique is used. In the second step, a feature-based generalized regression neural network is applied to track the detected objects throughout the frames in the video sequence. The proposed approach uses the centroid and area features of the moving objects and creates the reference regions instantly by selecting the objects within a circle. Simulation experiments are carried out on several turbulence-degraded video sequences and comparisons with an earlier method confirms that the proposed approach provides a more effective tracking of the targets.     

AN OBJECT-ORIENTED ARCHITECTURE FOR A FINITE ELEMENT METHOD KNOWLEDGE-BASED SYSTEM

A finite element method (FEM) system is complex in nature and still faces bottleneck problems in its maintenance, extension, etc., and it is yet necessary to be dealt with using some new methodologies. An object-orientation is a promising paradigm for treating complexities. In this paper, first an object-oriented FEM knowledge base system architecture is proposed. Then, an object model in the FEM domain is established through entity abstractions, action abstractions, category abstractions, agent abstractions, etc. Finally, a semantic network results from a semantic analysis of the FEM object model to represent the generative relationships among the FEM objects. Through an illustrative example, it is shown that a control task of finite element elasto-static analysis can be represented by a traveling path in the semantic network.     

Selection of keyframes for video colourization using steerable filtering

The appropriate selection of distinctive keyframes to represent the salient contents of a video is a critical task in video processing applications that rely on content analysis or information retrieval. Although many of the existing keyframe selection techniques perform satisfactorily in capturing salient visual contents, they often fail to adequately highlight the changes in visual information brought about by motion of objects between frames. In this paper, we propose a technique for keyframe selection by formulating the dissimilarity between the frames of a video shot in terms of the change in orientations that the corresponding objects of the two frames have undergone due to motion. This is accomplished by steerable filtering of the frames in order to extract the information about the local orientation of pixels within each frame. The frame to frame dissimilarity is adaptively thresholded over a group of frames in order to select the keyframes. In essence, keyframes are selected at the temporal instances where the change in orientation attains local maxima. Our keyframe selection methodology is specifically relevant to video colourization due to the fact that the keyframes that are to be employed for colourization must be chosen such that they capture all orientational changes effectively, while ensuring adequate content coverage.     

Predicting Simplified Thematic Progression Pattern for Discourse Analysis

The pattern of thematic progression, reflecting the semantic relationships between contextual two sentences, is an important subject in discourse analysis. We introduce a new corpus of Chinese news discourses annotated with thematic progression information and explore some computational methods to automatically extracting the discourse structural features of simplified thematic progression pattern (STPP) between contextual sentences in a text. Furthermore, these features are used in a hybrid approach to a major discourse analysis task, Chinese coreference resolution. This novel approach is built up via heuristic sieves and a machine learning method that comprehensively utilizes both the top-down STPP features and the bottom-up semantic features. Experimental results on the intersection of the CoNLL-2012 task shared dataset and the CDTC corpus demonstrate the effectiveness of our proposed approach.     

快速背景重建的在线运动目标检测

为了能快速地从视频图像序列中创建可靠的背景图像,进而提取运动目标,文中提出了一种基于反馈信息的运动目标检测算法.首先提出了基于相邻帧信息和背景估计信息相融合的背景重建算法,保证了在视频场景改变时仍能迅速捕捉背景;还提出了基于一种在线Otsu法的运动目标检测,将相邻帧运动目标信息反馈到目标提取算法中,弥补传统Otsu法的不足;最后提出了对光线变化具有一定鲁棒性的背景估计算法.实验表明,该方法的重建速度快,准确率高,能满足实时检测的需要.     

Data input model for virtual reality-aided facility layout

An approach to automatically extract three dimensional (3D) models (that is, geometries and topologies) of physical objects in a facility is described. The rationale for this work is its repeated use in efficiently developing databases of 3D objects for applying virtual reality (VR) tools in detailed layout decision support. Obtaining 3D object models can be a challenging task. Sometimes they are available, for example, in a Computer-Aided Design (CAD) database and these can be readily imported into a VR database. But on many occasions one is not so fortunate and these object models have to be created in correlation to an existing or proposed facility, which can be an extremely tedious and time consuming task. A time efficient and economical alternative is to use video camera images, but quickly and accurately capturing the depth information from 2D camera images has so far remained elusive because the existing methodologies are too general purpose and operate at a lower level of abstraction, namely digitized images. We have developed a method for directly inputting 3D objects into VR-aided facility layout models, by integrating the strengths of previously tried and tested technological components: (i) camera calibration; (ii) image processing; (iii) stereo vision; and (iv) Delaunay triangulation. The techniques described here are embedded in a prototype architecture and toolkit called MIRRORS (Methodology for Inputting Raw Recordings into 3D Object Renderings for Stereo). The primary contribution of this paper is that it has been able to design an integrated system to build 3D object models from 2D images. The MIRRORS system has been primarily designed for objects without free-form surfaces and whose shape can be recovered from a relatively nondense set of points.     

基于差异积累的视频运动对象自动分割

针对视频运动对象的自动分割,本文给出了一种基于差异积累的自动分割算法。与传统的基于运动信息变化检测方法不同,该算法通过累积的帧差信息构建出可靠的背景,与当前帧比较进而提取出视频运动对象。本文提出了一种增强的基于Otsu法的自适应阈值化方法,能更准确地对背景差图像进行阈值化分割,克服了传统Otsu法阈值化容易失效的问题。改进的基于区域生长的定位方法更能避免传统方法的误定位及重定位的问题。实验结果表明,本文算法具有较好的实时性、自适应性以及鲁棒性,可以较为可靠地建立背景模型并进行实时更新,适用于刚体或非刚体存在平缓的光照变化以及摄像头微抖动的视频运动对象的自动分割。     

Semantic linkages in research information systems as a new data source for scientometric studies

A growing number of research information systems use a semantic linkage technique to represent in explicit mode information about relationships between elements of its content. This practice is coming nowadays to a maturity when already existed data on semantically linked research objects and expressed by this scientific relationships can be recognized as a new data source for scientometric studies. Recent activities to provide scientists with tools for expressing in a form of semantic linkages their knowledge, hypotheses and opinions about relationships between available information objects also support this trend. The study presents one of such activities performed within the Socionet research information system with a special focus on (a) taxonomy of scientific relationships, which can exist between research objects, especially between research outputs; and (b) a semantic segment of a research e-infrastructure that includes a semantic interoperability support, a monitoring of changes in linkages and linked objects, notifications and a new model of scientific communication, and at last—scientometric indicators built by processing of semantic linkages data. Based on knowledge what is a semantic linkage data and how it is stored in a research information system we propose an abstract computing model of a new data source. This model helps with better understanding what new indicators can be designed for scientometric studies. Using current semantic linkages data collected in Socionet we present some statistical experiments, including examples of indicators based on two data sets: (a) what objects are linked and (b) what scientific relationships (semantics) are expressed by the linkages.     

Motion tracking system in video based on extensive feature set

In recent years, object detection and tracking has been a dynamic research area. Rapid development of the multimedia and the associated technologies urge the processing of a huge database of video clips. The processing efficiency lies on the search methodologies utilised in the video processing system. Usage of unsuitable search methodologies may make the processing system ineffective. Hence, effective object detection and tracking system is an essential criterion for searching relevant videos from a huge collection of videos. This paper proposes a unique object detection and tracking system where video segmentation, feature extraction, object detection and tracking are combined perfectly using various features. Initially, the database video clips are segmented into different shots before performing the feature extraction process. The proposed system consists of two stages, namely, feature extraction and tracking of object in the video clips. In the feature extraction stage, firstly, colour feature is extracted based on colour quantisation. Next, edge density feature is extracted for the objects present in the query video. Then, the texture feature is extracted based on LGXP technique. Finally, based on these feature extracted, the object will be detected and the detected objects will be tracked by utilising both forward and backward tracking technique. The proposed methodology proved to be more effective and accurate in object detection and tracking.     

Spatio-temporal motion-based foreground segmentation and shadow suppression

A relevant problem in computer vision is how to detect and track moving objects from video sequences efficiently. Some algorithms require manual calibration in terms of specification of parameters or some hypotheses. A novel method is developed to extract moving objects through multi-scale wavelet transform across background subtraction. The optimal selection of threshold is automatically determined which does not require any complex supervised training or manual calibration. The proposed approach is efficient in detecting moving objects with low contrast against the background and the detection is less affected by the presence of moving objects in the scene. The developed method combines region connectivity with chromatic consistency to overcome the aperture problem. Ghosts are removed by the proposed background update function, which efficiently prevents undesired corruption of background model and does not consider adaptation coefficient. The mentioned approach is scene-independent and the capacity to extract moving object and suppress cast shadow is high. The developed algorithm is flexible and computationally cost-effective. Experiments show that the proposed approach is robust and efficient in segmenting foreground and suppressing shadow by comparison.     

一种视频图像序列中运动对象的分割与跟踪算法

本文提出了一种视频图像序列中运动对象的分割与跟踪算法。该算法通过Canny算子检测出差帧图像的边缘信息,并结合当前帧与背景帧的边缘图像,提取出运动对象。在后续帧中通过建立前帧感兴趣运动对象与当前帧中各运动对象的帧间向量来跟踪当前帧中感兴趣的视频对象。实验结果表明,该算法可行,而且由于该算法简单、计算复杂度小,能很好地满足实时监控系统中对感兴趣运动对象的提取与跟踪。     

Video object inpainting: a scale-robust method

Abstract

Video inpainting is the process of reconstructing damaged regions of corrupted frames. In this research, we raise a few issues in existing video inpainting systems. They are usually not robust to the change in the object scale and cannot handle large missing regions behind the moving object. In this attempt, we will address the above issues as following: first, we extract moving objects from the background and construct two mosaic images for each object, a small mosaic and a large mosaic image. The small mosaic is used to detect the amount of scale changes in the moving objects and the large one is used to inpaint partially or completely corrupted objects. We next place the inpainted moving foreground in its location and rescale the objects to their original scale. Finally, we combine the inpainted moving foreground and the background to obtain the corrected video. To speed up the process, we have utilised a multi-resolution approach so that the patch are initially matched in a coarse resolution and later are refined in a fine resolution. The results confirm the robustness of our method in handling the scale change of moving objects and large missing regions.     

Multi-level trajectory learning for traffic behavior detection and analysis

Motion patterns can be learnt automatically based on object trajectories data extracted by means of video tracking, which is an effective approach for modeling and analyzing traffic behavior. In this paper, a multi-level motion pattern learning approach for traffic behavior analysis is presented, which takes into account the spatial characteristics, direction characteristics, and type characteristics of trajectories. At the spatial level, improved Hausdorff distance measurement is applied to construct a spatial similarity matrix of the trajectories collected, and spectral clustering is used to achieve spatial pattern learning. At the directional level, the start and end points of trajectories are fitted using a Gaussian mixed model to extract the distribution of entry and exit zones. Then, the direction pattern is obtained from the regional centers of the pairwise distribution zones. At the type level, the type pattern is acquired through a K-means clustering algorithm that considers multiple classification features of trajectories. Based on the learned multi-level motion patterns, abnormal behavior detection algorithms are further developed by means of pattern matching. Finally, our approach is tested with several video sequences from real-world traffic scenarios. Some typical traffic behaviors in the test scenarios are successfully recognized and analyzed and examples of abnormal traffic behaviors are also reliably detected.