Automated and coupled services of advanced smart surveillance systems toward green IT: tracking,retrieval and digital evidence

Green Security is a new research field defining and investigating security solutions using an energy-aware perspective. Growing efforts and interests for an intelligent or smart surveillance system which is capable of automatically detecting and tracking target objects is in the spotlight in the security community. So far, these technologies are mainly aimed at single camera applications and are evolving with the demand for wide-area surveillance systems currently. However, the tracking techniques used on a single camera have limitations in providing effective crime prevention and countermeasures when an incident occurs since an object is not linked to other cameras. In addition, the use of multi-camera systems for wide-area surveillance not only produces large amounts of video data to be stored, but also have more technical requirements in the interrelation between cameras or server. It require a considerable amount of time, manpower and energy in multi-camera tracking and back-tracking of objects. Therefore, we propose the advanced smart surveillance system for wide-areas which is capable of the automated tracking and retrieval of target object and digital evidence-video collection. Furthermore, we considered the multiple-camera environment with non-overlapping views which includes more constraint conditions by various light changes. This system enables real-time object tracking, fast post-retrieval and selective digital evidence collection with economy of time, manpower, memory devices, and energy consumption. Also, this system is more energy-efficient since our schemes are organically connected to each other.     

Inference topology of distributed camera networks with multiple cameras

This paper proposes an inference method to construct the topology of a camera network with overlapping and non-overlapping fields of view for a commercial surveillance system equipped with multiple cameras. It provides autonomous object detection, tracking and recognition in indoor or outdoor urban environments. The camera network topology is estimated from object tracking results among and within FOVs. The merge-split method is used for object occlusion in a single camera and an EM-based approach for extracting the accurate object feature to track moving people and establishing object correspondence across multiple cameras. The appearance of moving people and the transition time between entry and exit zones is measured to track moving people across blind regions of multiple cameras with non-overlapping FOVs. Our proposed method graphically represents the camera network topology, as an undirected weighted graph using the transition probabilities and 8-directional chain code. The training phase and the test were run with eight cameras to evaluate the performance of our method. The temporal probability distribution and the undirected weighted graph are shown in the experiments.     

Camera handoff with adaptive resource management for multi-camera multi-object tracking

Camera handoff is a crucial step to obtain a continuously tracked and consistently labeled trajectory of the object of interest in multi-camera surveillance systems. Most existing camera handoff algorithms concentrate on data association, namely consistent labeling, where images of the same object are identified across different cameras. However, there exist many unsolved questions in developing an efficient camera handoff algorithm. In this paper, we first design a trackability measure to quantitatively evaluate the effectiveness of object tracking so that camera handoff can be triggered timely and the camera to which the object of interest is transferred can be selected optimally. Three components are considered: resolution, distance to the edge of the camera’s field of view (FOV), and occlusion. In addition, most existing real-time object tracking systems see a decrease in the frame rate as the number of tracked objects increases. To address this issue, our handoff algorithm employs an adaptive resource management mechanism to dynamically allocate cameras’ resources to multiple objects with different priorities so that the required minimum frame rate is maintained. Experimental results illustrate that the proposed camera handoff algorithm can achieve a substantially improved overall tracking rate by 20% in comparison with the algorithm presented by Khan and Shah.     

仿生复眼式全景探测及跟踪策略

为解决传统相机不能在全景范围内快速连续跟踪运动物体的弊端,本文提出一种仿生复眼式全景探测思想,以及配套的跟踪策略。在硬件方面,采用多个子眼相机进行全景实时探测,中心采用一个大孔径主眼相机,并且将其安装于云台上面,通过云台控制器接收的不同指令码进行精确定位。在软件方面,本文采用自动加窗采集的思想,对于运动区域的图像进行多层窗口采集,非运动区域则减少窗口层数,然后将采集的图像进行高斯运动目标检测,而检测方法本文在重叠区采用了仿生复眼的侧抑制算法,得到了比较清晰的运动目标轮廓。按照上述软硬件思想本文搭建了实际实验装置,并且进行了重复性实验。由于实时探测特点,以及加窗采集和侧抑制算法的结合,相比传统相机的跟踪效果,在视场和灵敏度上得到了很大的提高。     

An efficient continuous tracking system in real-time surveillance application

A tracking object must present a proper field of view (FOV) in a multiple active camera surveillance system; its clarity can facilitate smooth processing by the surveillance system before further processing, such as face recognition. However, when pan–tilt–zoom (PTZ) cameras are used, the tracking object can be brought into the FOV by adjusting its intrinsic parameters; consequently, selection of the best-performing camera is critical. Performance is determined by the relative positions of the camera and the tracking objects, image quality, lighting and how much of the front side of the object faces the camera. In a multi-camera surveillance system, both camera hand-off and camera assignment play an important role in automated and persistent tracking, which are typical surveillance requirements. This study investigates the use of automatic methods for tracking an object across cameras in a surveillance network using PTZ cameras. An automatic, efficient continuous tracking scheme is developed. The goal is to determine the decision criteria for hand-off using Sight Quality Indication (SQI) (which includes information on the position of the object and the proportion of the front of object faces the camera), and to perform the camera handoff task in a manner that optimizes the vision effect associated with monitoring. Experimental results reveal that the proposed algorithm can be efficiently executed, and the handoff method for feasible and continuously tracking active objects under real-time surveillance.     

Adaptive online camera coordination for multi-camera multi-target surveillance

Online camera selection is introduced as a result of the improved mobility of cameras and the increased scale of surveillance systems. Most existing camera assignment algorithms achieve an optimal observation under the assumption of the unlimited camera computational capacities. However, practical surveillance systems experience resource limitation and see a degradation in the system performance as the number of objects to be processed increases. To address this issue, we propose an adaptive camera assignment algorithm considering the limited camera computational capacities. In so doing, camera resources can be dynamically allocated to multiple objects according to their priorities and the current camera computational load. Experimental results illustrate that the proposed camera assignment algorithm is capable of maintaining a constant frame rate and achieving a substantially decreased object rejection rate in comparison with the algorithm presented by Bakhtari and Benhabib.     

Active Tuning of Intrinsic Camera Parameters

In the last years, the research effort of the scientific community to study systems for ambient intelligence has been really strong. Usually, the systems developed so far base their analysis on images acquired by automatic cameras. In this paper, we propose a way to develop new smart systems that are able to actively decide both what to see and how to see it. In particular, the main idea is to tune the acquisition parameters on the basis of what the system desires to acquire. The regulation strategy is based on two camera parameters, focus and iris. It aims to identify an optimal sequence of steps to enhance the acquisition quality of an object of interest. To this end, a hierarchy of neural networks has been employed first to select which parameter must be regulated then to adjust it. The proposed solution can be applied to both static and moving cameras. The results show how the proposed technique can be applied to images acquired by a moving camera with zoom capabilities for surveillance purposes.     

Discriminative learning and recognition of image set classes using canonical correlations

We address the problem of comparing sets of images for object recognition, where the sets may represent variations in an object's appearance due to changing camera pose and lighting conditions. canonical correlations (also known as principal or canonical angles), which can be thought of as the angles between two d-dimensional subspaces, have recently attracted attention for image set matching. Canonical correlations offer many benefits in accuracy, efficiency, and robustness compared to the two main classical methods: parametric distribution-based and nonparametric sample-based matching of sets. Here, this is first demonstrated experimentally for reasonably sized data sets using existing methods exploiting canonical correlations. Motivated by their proven effectiveness, a novel discriminative learning method over sets is proposed for set classification. Specifically, inspired by classical linear discriminant analysis (LDA), we develop a linear discriminant function that maximizes the canonical correlations of within-class sets and minimizes the canonical correlations of between-class sets. Image sets transformed by the discriminant function are then compared by the canonical correlations. Classical orthogonal subspace method (OSM) is also investigated for the similar purpose and compared with the proposed method. The proposed method is evaluated on various object recognition problems using face image sets with arbitrary motion captured under different illuminations and image sets of 500 general objects taken at different views. The method is also applied to object category recognition using ETH-80 database. The proposed method is shown to outperform the state-of-the-art methods in terms of accuracy and efficiency     

Illumination normalization with time-dependent intrinsic images for video surveillance

Variation in illumination conditions caused by weather, time of day, etc., makes the task difficult when building video surveillance systems of real world scenes. Especially, cast shadows produce troublesome effects, typically for object tracking from a fixed viewpoint, since it yields appearance variations of objects depending on whether they are inside or outside the shadow. In this paper, we handle such appearance variations by removing shadows in the image sequence. This can be considered as a preprocessing stage which leads to robust video surveillance. To achieve this, we propose a framework based on the idea of intrinsic images. Unlike previous methods of deriving intrinsic images, we derive time-varying reflectance images and corresponding illumination images from a sequence of images instead of assuming a single reflectance image. Using obtained illumination images, we normalize the input image sequence in terms of incident lighting distribution to eliminate shadowing effects. We also propose an illumination normalization scheme which can potentially run in real time, utilizing the illumination eigenspace, which captures the illumination variation due to weather, time of day, etc., and a shadow interpolation method based on shadow hulls. This paper describes the theory of the framework with simulation results and shows its effectiveness with object tracking results on real scene data sets.     

Can You See Me Now? Sensor Positioning for Automated and Persistent Surveillance

Most existing camera placement algorithms focus on coverage and/or visibility analysis, which ensures that the object of interest is visible in the camera's field of view (FOV). However, visibility, which is a fundamental requirement of object tracking, is insufficient for automated persistent surveillance. In such applications, a continuous consistently labeled trajectory of the same object should be maintained across different camera views. Therefore, a sufficient uniform overlap between the cameras' FOVs should be secured so that camera handoff can successfully and automatically be executed before the object of interest becomes untraceable or unidentifiable. In this paper, we propose sensor-planning methods that improve existing algorithms by adding handoff rate analysis. Observation measures are designed for various types of cameras so that the proposed sensor-planning algorithm is general and applicable to scenarios with different types of cameras. The proposed sensor-planning algorithm preserves necessary uniform overlapped FOVs between adjacent cameras for an optimal balance between coverage and handoff success rate. In addition, special considerations such as resolution and frontal-view requirements are addressed using two approaches: 1) direct constraint and 2) adaptive weights. The resulting camera placement is compared with a reference algorithm published by Erdem and Sclaroff. Significantly improved handoff success rates and frontal-view percentages are illustrated via experiments using indoor and outdoor floor plans of various scales.      

基于宽容训练和隐私保护的快速监控视频检索模型

监控视频关键帧检索和属性查找在交通、安防、教育等领域具有众多应用场景,应用深度学习模型处理海量视频数据在一定程度上缓解了人力消耗,但是存在隐私泄露、计算资源消耗大、时间长等特点.基于上述场景,提出了一个面向大规模监控视频的安全、快速的视频检索模型.具体地,根据云端算力大、监控摄像头算力规模小的特点,在云端部署重量级模型,并使用所提出的宽容训练策略对其进行定制化知识蒸馏,将蒸馏后的轻量级模型部署在监控摄像头内,同时使用局部加密算法对图像敏感部分进行加密,结合云端TEE技术和用户授权机制,在极低资源消耗的情况下实现隐私保护.通过合理控制蒸馏策略的“容忍度”,能够较好地平衡摄像头视频输入阶段和云端检索阶段的耗时,在保证极高准确率的前提下,保证极低的检索时延.相比于传统检索方法,该模型具有安全高效、可伸缩、低延时的特点.实验结果显示,在多个公开数据集上,该模型相比于传统检索方法提供9×-133×的加速.     

Camera handoff and placement for automated tracking systems with multiple omnidirectional cameras

In a multi-camera surveillance system, both camera handoff and placement play an important role in generating an automated and persistent object tracking, typical of most surveillance requirements. Camera handoff should comprise three fundamental components, time to trigger handoff process, the execution of consistent labeling, and the selection of the next optimal camera. In this paper, we design an observation measure to quantitatively formulate the effectiveness of object tracking so that we can trigger camera handoff timely and select the next camera appropriately before the tracked object falls out of the field of view (FOV) of the currently observing camera. In the meantime, we present a novel solution to the consistent labeling problem in omnidirectional cameras. A spatial mapping procedure is proposed to consider both the noise inherent to the tracking algorithms used by the system and the lens distortion introduced by omnidirectional cameras. This does not only avoid the tedious process, but also increases the accuracy, to obtain the correspondence between omnidirectional cameras without human interventions. We also propose to use the Wilcoxon Signed-Rank Test to improve the accuracy of trajectory association between pairs of objects. In addition, since we need a certain amount of time to successfully carry out the camera handoff procedure, we introduce an additional constraint to optimally reserve sufficient cameras’ overlapped FOVs for the camera placement. Experiments show that our proposed observation measure can quantitatively formulate the effectiveness of tracking, so that camera handoff can smoothly transfer objects of interest. Meanwhile, our proposed consistent labeling approach can perform as accurately as the geometry-based approach without tedious calibration processes and outperform Calderara’s homography-based approach. Our proposed camera placement method exhibits a significant increase in the camera handoff success rate at the cost of slightly decreased coverage, as compared to Erdem and Sclaroff’s method without considering the requirement on overlapped FOVs.     

Shape-based indexing scheme for camera view invariant 3-D object retrieval

Camera view invariant 3-D object retrieval is an important issue in many traditional and emerging applications such as security, surveillance, computer-aided design (CAD), virtual reality, and place recognition. One straightforward method for camera view invariant 3-D object retrieval is to consider all the possible camera views of 3-D objects. However, capturing and maintaining such views require an enormous amount of time and labor. In addition, all camera views should be indexed for reasonable retrieval performance, which requires extra storage space and maintenance overhead. In the case of shape-based 3-D object retrieval, such overhead could be relieved by considering the symmetric shape feature of most objects. In this paper, we propose a new shape-based indexing and matching scheme of real or rendered 3-D objects for camera view invariant object retrieval. In particular, in order to remove redundant camera views to be indexed, we propose a camera view skimming scheme, which includes: i) mirror shape pairing and ii) camera view pruning according to the symmetrical patterns of object shapes. Since our camera view skimming scheme considerably reduces the number of camera views to be indexed, it could relieve the storage requirement and improve the matching speed without sacrificing retrieval accuracy. Through various experiments, we show that our proposed scheme can achieve excellent performance.     

基于模糊集合的汉语主观句识别

主观句识别的工作在诸如情感分类和意见摘要等意见挖掘系统中占有很重要的地位。在该文中,我们提出一种基于情感密度的模糊集合分类器以识别汉语主观句。首先,我们利用优势率方法从训练语料中抽取主观性线索词;然后,为了能更好的表达一个句子的主观性,我们利用抽取出的主观性线索词计算出每个句子的情感密度;最后,我们结合情感密度的特点实现了一个三角形隶属度函数的模糊集合分类器以识别主观句。我们在NTCIR-6中文数据中做了两组实验。实验结果表明我们的方法具有一定的可行性。     

基于域适应的X光图像的目标检测

随着卷积神经网络的发展,X光安全检查图像的自动目标检测算法已经取得了重大进步.但是,当将这些目标检测算法应用到不同于训练集数据的新数据,即训练域数据和测试域数据的图像数据服从不一致的分布时,这些检测算法的性能通常会降低.根据X光成像的变化,提出一种基于上下文的透射率自适应域对齐方法,用于解决检测算法的域不适应问题.首先...     

Camera cooperation for achieving visual attention

In this paper we address the problem of establishing a computational model for visual attention using cooperation between two cameras. More specifically we wish to maintain a visual event within the field of view of a rotating and zooming camera through the understanding and modeling of the geometric and kinematic coupling between a static camera and an active camera. The static camera has a wide field of view thus allowing panoramic surveillance at low resolution. High-resolution details may be captured by a second camera, provided that it looks in the right direction. We derive an algebraic formulation for the coupling between the two cameras and we specify the practical conditions yielding a unique solution. We describe a method for separating a foreground event (such as a moving object) from its background while the camera rotates. A set of outdoor experiments shows the two-camera system in operation.     

Object matching in disjoint cameras using a color transfer approach

Object appearance models are a consequence of illumination, viewing direction, camera intrinsics, and other conditions that are specific to a particular camera. As a result, a model acquired in one view is often inappropriate for use in other viewpoints. In this work we treat this appearance model distortion between two non-overlapping cameras as one in which some unknown color transfer function warps a known appearance model from one view to another. We demonstrate how to recover this function in the case where the distortion function is approximated as general affine and object appearance is represented as a mixture of Gaussians. Appearance models are brought into correspondence by searching for a bijection function that best minimizes an entropic metric for model dissimilarity. These correspondences lead to a solution for the transfer function that brings the parameters of the models into alignment in the UV chromaticity plane. Finally, a set of these transfer functions acquired from a collection of object pairs are generalized to a single camera-pair-specific transfer function via robust fitting. We demonstrate the method in the context of a video surveillance network and show that recognition of subjects in disjoint views can be significantly improved using the new color transfer approach.     

Tracking non-rigid objects using probabilistic Hausdorff distance matching

This paper proposes a new method of extracting and tracking a non-rigid object moving against a cluttered background while allowing camera movement. For object extraction we first detect an object using watershed segmentation technique and then extract its contour points by approximating the boundary using the idea of feature point weighting. For object tracking we take the contour to estimate its motion in the next frame by the maximum likelihood method. The position of the object is estimated using a probabilistic Hausdorff measurement while the shape variation is modelled using a modified active contour model. The proposed method is highly tolerant to occlusion. Unless an object is fully occluded during tracking, the result is stable and the method is robust enough for practical application.