高速公路云台相机的自动标定

目的 云台相机因监控视野广、灵活度高,在高速公路监控系统中发挥出重要的作用,但因云台相机焦距与角度不定时地随监控需求变化,对利用云台相机的图像信息获取真实世界准确的物理信息造成一定困难,因此进行云台相机非现场自动标定方法的研究对高速公路监控系统的应用具有重要价值。方法 本文提出了一种基于消失点约束与车道线模型约束的云台相机自动标定方法,以建立高速公路监控系统的图像信息与真实世界物理信息之间准确描述关系。首先,利用车辆目标运动轨迹的级联霍夫变换投票实现纵向消失点的准确估计,其次以车道线模型物理度量为约束,并采用枚举策略获取横向消失点的准确估计,最终在已知相机高度的条件下实现高速公路云台相机标定参数的准确计算。结果 将本文方法在不同的场景下进行实验,得到在不同的距离下的平均误差分别为4.63%、4.74%、4.81%、4.65%,均小于5%。结论 对多组高速公路监控场景的测试实验结果表明,本文提出的云台相机自动标定方法对高速公路监控场景的物理测量误差能够满足应用需求,与参考方法相比较而言具有较大的优势和一定的应用价值,得到的相机内外参数可用于计算车辆速度与空间位置等。     

Dynamic calibration of pan-tilt-zoom cameras for traffic monitoring.

Pan-tilt-zoom (PTZ) cameras have been widely used in recent years for monitoring and surveillance applications. These cameras provide flexible view selection as well as a wider observation range. This makes them suitable for vision-based traffic monitoring and enforcement systems. To employ PTZ cameras for image measurement applications, one first needs to calibrate the camera to obtain meaningful results. For instance, the accuracy of estimating vehicle speed depends on the accuracy of camera calibration and that of vehicle tracking results. This paper presents a novel calibration method for a PTZ camera overlooking a traffic scene. The proposed approach requires no manual operation to select the positions of special features. It automatically uses a set of parallel lane markings and the lane width to compute the camera parameters, namely, focal length, tilt angle, and pan angle. Image processing procedures have been developed for automatically finding parallel lane markings. Interesting experimental results are presented to validate the robustness and accuracy of the proposed method.     

双摄像机协同人脸鹰眼检测与定位方法

现代视频监控系统需要获取大范围场景中感兴趣目标的清晰图像,这在目标距离较远并且不断移动时单纯采用摄像机调焦方式通常有一定的困难。为了获取宽范围监控场景中远距离行人的主要面部特征,采用广角静止—窄视场运动双摄像机协同工作方式可以同时获得远距离目标的全局和细节信息。首先采用改进的Codebook背景减法从广角摄像机中检测运动目标,然后指引运动摄像机近距离跟踪观察;若行人停止运动,则利用运动摄像机对其进行放大,然后从中检测人脸,并将人脸置于视野中心放大得到清晰图像。当行人再次运动时,广角相机将初始位置再次传递给运动摄像机,由其再对行人进行跟踪。通过实验室内和室外真实场景的实验表明,广角相机的检测算法具有一定的鲁棒性,运动相机能跟踪放大行人人脸图像,算法运行速度满足实时性要求。     

Dynamic Calibration of Pan–Tilt–Zoom Cameras for Traffic Monitoring

Pan–tilt–zoom (PTZ) cameras have been widely used in recent years for monitoring and surveillance applications. These cameras provide flexible view selection as well as a wider observation range. This makes them suitable for vision-based traffic monitoring and enforcement systems. To employ PTZ cameras for image measurement applications, one first needs to calibrate the camera to obtain meaningful results. For instance, the accuracy of estimating vehicle speed depends on the accuracy of camera calibration and that of vehicle tracking results. This paper presents a novel calibration method for a PTZ camera overlooking a traffic scene. The proposed approach requires no manual operation to select the positions of special features. It automatically uses a set of parallel lane markings and the lane width to compute the camera parameters, namely, focal length, tilt angle, and pan angle. Image processing procedures have been developed for automatically finding parallel lane markings. Interesting experimental results are presented to validate the robustness and accuracy of the proposed method.     

摄像机运动情况下的运动对象检测

在监控应用中,由于场景是已知的,因此可以使用背景减去法检测运动对象.当摄像机进行扫描和倾斜运动时,需要使用多个图像帧才能完整地表示监控场景.如何组织和索引这些背景帧属于摄像机跟踪问题.提出一种无需摄像机标定的背景帧索引和访问方法.这一方法需要使用图像配准技术估计图像初始运动参数.提出一种屏蔽外点的图像配准算法,综合利用线性回归和稳健回归快速估计初始运动参数.为了快速计算连续帧之间的运动参数,提出一种基于四参数模型的优化算法.利用非参数背景维护模型抑制虚假运动象素.室内和户外实验结果表明本文方法是有效的.     

Continuous localization and mapping of a pan–tilt–zoom camera for wide area tracking

Pan–tilt–zoom (PTZ) cameras are well suited for object identification and recognition in far-field scenes. However, the effective use of PTZ cameras is complicated by the fact that a continuous online camera calibration is needed and the absolute pan, tilt and zoom values provided by the camera actuators cannot be used because they are not synchronized with the video stream. So, accurate calibration must be directly extracted from the visual content of the frames. Moreover, the large and abrupt scale changes, the scene background changes due to the camera operation and the need of camera motion compensation make target tracking with these cameras extremely challenging. In this paper, we present a solution that provides continuous online calibration of PTZ cameras which is robust to rapid camera motion, changes of the environment due to varying illumination or moving objects. The approach also scales beyond thousands of scene landmarks extracted with the SURF keypoint detector. The method directly derives the relationship between the position of a target in the ground plane and the corresponding scale and position in the image and allows real-time tracking of multiple targets with high and stable degree of accuracy even at far distances and any zoom level.     

Panoramic Gaussian Mixture Model and large-scale range background substraction method for PTZ camera-based surveillance systems

In this paper, we present a novel approach for constructing a large-scale range panoramic background model that provides fast registration of the observed frame and localizes the foreground targets with arbitrary camera direction and scale in a Pan–tilt–zoom (PTZ) camera-based surveillance system. Our method consists of three stages. (1) In the first stage, a panoramic Gaussian mixture model (PGMM) of the PTZ camera’s field of view is generated off-line for later use in on-line foreground detection. (2) In the second stage, a multi-layered correspondence ensemble is generated off-line from frames captured at different scales which is used by the correspondence propagation method to register observed frames online to the PGMM. (3) In the third stage, foreground is detected and the PGMM is updated. The proposed method has the capacity to deal with the PTZ camera’s ability to cover a wide field of view (FOV) and large-scale range. We demonstrate the advantages of the proposed PGMM background subtraction method by incorporating it with a tracking system for surveillance applications.     

Systems and algorithms for autonomous and scalable crowd surveillance using robotic PTZ cameras assisted by a wide-angle camera

We report an autonomous surveillance system with multiple pan-tilt-zoom (PTZ) cameras assisted by a fixed wide-angle camera. The wide-angle camera provides large but low resolution coverage and detects and tracks all moving objects in the scene. Based on the output of the wide-angle camera, the system generates spatiotemporal observation requests for each moving object, which are candidates for close-up views using PTZ cameras. Due to the fact that there are usually much more objects than the number of PTZ cameras, the system first assigns a subset of the requests/objects to each PTZ camera. The PTZ cameras then select the parameter settings that best satisfy the assigned competing requests to provide high resolution views of the moving objects. We propose an approximation algorithm to solve the request assignment and the camera parameter selection problems in real time. The effectiveness of the proposed system is validated in both simulation and physical experiment. In comparison with an existing work using simulation, it shows that in heavy traffic scenarios, our algorithm increases the number of observed objects by over 210%.     

Persistent people tracking and face capture using a PTZ camera

Pan–tilt–zoom (PTZ) camera is a powerful tool in far-field scenarios. However, most of the current PTZ surveillance systems require manual intervention to move the camera to the desired position. In this paper, we address the problem of persistent people tracking and face capture in uncontrolled scenarios using a single PTZ camera, which could prove most helpful in forensic applications. The system first detects and tracks pedestrians in zoomed-out mode. Then, according to a scheduler, the system selects a person to zoom in. In the zoomed-in mode, we detect a set of face images and solve the face–face association and face–person association problems. The system then zooms back out where tracking is continued as people re-appear in the view. The person–person association module associates the people on the schedule list with the people in the current view. The detected faces are associated with the corresponding people and trajectories. Due to the dynamic nature of our problem, e.g. the field of view of the camera changes because of the pan/tilt/zoom movement of the camera, all of the processes including receiving images from the camera and processing must be done in real time. To the best of our knowledge, the proposed method is the first to address the association of face images to people and trajectories using a single PTZ camera. Extensive experiments in challenging indoor and outdoor uncontrolled conditions demonstrate the effectiveness of the proposed system.     

基于相机标定的跨相机场景拼接方法

针对单相机场景下的智能交通应用得到了较好的发展,但跨区域研究还处在起步阶段的问题,本文提出一种基于相机标定的跨相机场景拼接方法.首先,通过消失点标定分别建立两个相机场景中子世界坐标系下的物理信息到二维图像的映射关系;其次通过两个子世界坐标系间的公共信息来完成相机间的投影变换;最后,通过提出的逆投影思想和平移矢量关系来完成道路场景拼接.通过实验结果表明,该方法能够较好的实现道路场景拼接和跨区域道路物理测量,为相关实际应用研究奠定基础.     

An adaptive focus-of-attention model for video surveillance and monitoring

In current video surveillance systems, commercial pan/tilt/zoom (PTZ) cameras typically provide naive (or no) automatic scanning functionality to move a camera across its complete viewable field. However, the lack of scene-specific information inherently handicaps these scanning algorithms. We address this issue by automatically building an adaptive, focus-of-attention, scene-specific model using standard PTZ camera hardware. The adaptive model is constructed by first detecting local human activity (i.e., any translating object with a specific temporal signature) at discrete locations across a PTZ camera’s entire viewable field. The temporal signature of translating objects is extracted using motion history images (MHIs) and an original, efficient algorithm based on an iterative candidacy-classification-reduction process to separate the target motion from noise. The target motion at each location is then quantified and employed in the construction of a global activity map for the camera. We additionally present four new camera scanning algorithms which exploit this activity map to maximize a PTZ camera’s opportunity of observing human activity within the camera’s overall field of view. We expect that these efficient and effective algorithms are implementable within current commercial camera systems.     

高速球形摄像机PTZ跟踪策略设计及实现

针对智能监控中基于高速球形摄像机的PTZ跟踪功能模块,设计了一种PTZ跟踪控制策略。该策略在球机机械参数未知的情况下,一方面能控制球机实时地跟踪目标使目标始终处于视野中央,另一方面可自动进行变倍动作来放大拍摄目标的局部细节。针对球机Zoom控制中跟踪窗口大小自适应调整的问题,利用SIFT算法设计了一种计算球机变倍率的方法。利用VS2005和OpenCV软件平台实现了PTZ跟踪的整体流程。实验表明,该策略能有效、稳定地进行PTZ跟踪。     

Dense 3-D Reconstruction of an Outdoor Scene by Hundreds-Baseline Stereo Using a Hand-Held Video Camera

Three-dimensional (3-D) models of outdoor scenes are widely used for object recognition, navigation, mixed reality, and so on. Because such models are often made manually with high costs, automatic 3-D reconstruction has been widely investigated. In related work, a dense 3-D model is generated by using a stereo method. However, such approaches cannot use several hundreds images together for dense depth estimation because it is difficult to accurately calibrate a large number of cameras. In this paper, we propose a dense 3-D reconstruction method that first estimates extrinsic camera parameters of a hand-held video camera, and then reconstructs a dense 3-D model of a scene. In the first process, extrinsic camera parameters are estimated by tracking a small number of predefined markers of known 3-D positions and natural features automatically. Then, several hundreds dense depth maps obtained by multi-baseline stereo are combined together in a voxel space.So, we can acquire a dense 3-D model of the outdoor scene accurately by using several hundreds input images captured by a hand-held video camera.     

分裂合并运动分割的多运动视觉里程计方法

目的 视觉里程计（visual odometry,VO）仅需要普通相机即可实现精度可观的自主定位,已经成为计算机视觉和机器人领域的研究热点,但是当前研究及应用大多基于场景为静态的假设,即场景中只有相机运动这一个运动模型,无法处理多个运动模型,因此本文提出一种基于分裂合并运动分割的多运动视觉里程计方法,获得场景中除相机运动外多个运动目标的运动状态。方法 基于传统的视觉里程计框架,引入多模型拟合的方法分割出动态场景中的多个运动模型,采用RANSAC（random sample consensus）方法估计出多个运动模型的运动参数实例;接着将相机运动信息以及各个运动目标的运动信息转换到统一的坐标系中,获得相机的视觉里程计结果,以及场景中各个运动目标对应各个时刻的位姿信息;最后采用局部窗口光束法平差直接对相机的姿态以及计算出来的相机相对于各个运动目标的姿态进行校正,利用相机运动模型的内点和各个时刻获得的相机相对于运动目标的运动参数,对多个运动模型的轨迹进行优化。结果 本文所构建的连续帧运动分割方法能够达到较好的分割结果,具有较好的鲁棒性,连续帧的分割精度均能达到近100%,充分保证后续估计各个运动模型参数的准确性。本文方法不仅能够有效估计出相机的位姿,还能估计出场景中存在的显著移动目标的位姿,在各个分段路径中相机自定位与移动目标的定位结果位置平均误差均小于6%。结论 本文方法能够同时分割出动态场景中的相机自身运动模型和不同运动的动态物体运动模型,进而同时估计出相机和各个动态物体的绝对运动轨迹,构建出多运动视觉里程计过程。     

Robust foreground detection in video using pixel layers

A framework for robust foreground detection that works under difficult conditions such as dynamic background and moderately moving camera is presented in this paper. The proposed method includes two main components: coarse scene representation as the union of pixel layers, and foreground detection in video by propagating these layers using a maximum-likelihood assignment. We first cluster into "layers" those pixels that share similar statistics. The entire scene is then modeled as the union of such non-parametric layer-models. An in-coming pixel is detected as foreground if it does not adhere to these adaptive models of the background. A principled way of computing thresholds is used to achieve robust detection performance with a pre-specified number of false alarms. Correlation between pixels in the spatial vicinity is exploited to deal with camera motion without precise registration or optical flow. The proposed technique adapts to changes in the scene, and allows to automatically convert persistent foreground objects to background and re-convert them to foreground when they become interesting. This simple framework addresses the important problem of robust foreground and unusual region detection, at about 10 frames per second on a standard laptop computer. The presentation of the proposed approach is complemented by results on challenging real data and comparisons with other standard techniques.     

基于场景模型与统计学习的鲁棒行人检测算法

提出一种基于场景模型和统计学习的行人检测算法. 针对训练行人检测器时面临的动态场景的复杂性和行人样本多样性等问题, 通过背景建模, 从场景的背景图像上提取有限的负样本用于训练, 大幅度提高了分类器的检测率, 同时降低了虚警; 提出一种快速弱分类器选择算法, 根据正、负样本特征大小的分布和期望的检测率, 直接求解特征大小的阈值范围, 能够满足在线训练和更新检测器的要求; 提出一种基于正样本错误率的训练算法, 先根据正样本加权错误率选择弱分类器, 快速提高检测率, 在训练结束后调整最终分类器的加权系数, 在保证检测率的同时尽可能降低虚警率. 实验中构建了一个试验视频数据库和行人样本库, 数据库包括雨、雪、阴影、季节变化、摄像机平移、旋转、缩放等情况, 并设计实现了一个实时行人检测系统BMAT (Background modeling and Adaboost training), 实验结果证明了算法的有效性.     

适用于旋转摄像下目标检测的图像补偿

目的 摄像机旋转扫描条件下的动目标检测研究中,传统的线性模型无法解决摄像机旋转扫描运动带来的图像间非线性变换问题,导致图像补偿不准确,在动目标检测时将引起较大误差,造成动目标虚假检测。为解决这一问题,提出了一种面阵摄像机旋转扫描条件下的图像补偿方法,其特点是能够同时实现背景运动补偿和图像非线性变换补偿,从而实现动目标的快速可靠检测。方法 首先进行图像匹配,然后建立摄像机旋转扫描非线性模型,通过参数空间变换将其转化为线性求解问题,采用Hough变换实现该方程参数的快速鲁棒估计。解决摄像机旋转扫描条件下获取的图像间非线性变换问题,从而实现图像准确补偿。在此基础上,可以利用帧间差分等方法检测出运动目标。结果 实验结果表明,在摄像机旋转扫描条件下,本文方法能够同时实现图像间的背景运动补偿和非线性变换补偿,可以去除大部分由于立体视差效应（parallax effects）产生的匹配错误。并且在实验中,本文方法处理速度可以达到50帧/s,满足实时性要求。结论 在面阵摄像机旋转扫描的条件下,相比于传统的基于线性模型的图像补偿方法,本文方法能够快速、准确地在背景补偿的基础上同时解决图像间非线性变换问题,从而更好地提取出运动目标,具有一定的实用价值。     

利用运动线索的单目深度测量

目的 传统的单目视觉深度测量方法具有设备简单、价格低廉、运算速度快等优点,但需要对相机进行复杂标定,并且只在特定的场景条件下适用。为此,提出基于运动视差线索的物体深度测量方法,从图像中提取特征点,利用特征点与图像深度的关系得到测量结果。方法 对两幅图像进行分割,获取被测量物体所在区域;然后采用本文提出的改进的尺度不变特征变换SIFT（scale-invariant feature transtorm）算法对两幅图像进行匹配,结合图像匹配和图像分割的结果获取被测量物体的匹配结果;用Graham扫描法求得匹配后特征点的凸包,获取凸包上最长线段的长度;最后利用相机成像的基本原理和三角几何知识求出图像深度。结果 实验结果表明,本文方法在测量精度和实时性两方面都有所提升。当图像中的物体不被遮挡时,实际距离与测量距离之间的误差为2.60%,测量距离的时间消耗为1.577 s;当图像中的物体存在部分遮挡时,该方法也获得了较好的测量结果,实际距离与测量距离之间的误差为3.19%,测量距离所需时间为1.689 s。结论 利用两幅图像上的特征点来估计图像深度,对图像中物体存在部分遮挡情况具有良好的鲁棒性,同时避免了复杂的摄像机标定过程,具有实际应用价值。     

立体图像对的生成

获取同一场景的立体图像对是实现双目立体成像的一个关键问题。提出了一种在三维场景已经建好的情况下生成立体图像对的方法。该方法根据双目立体视觉的原理，利用3DS MAX中的摄像机对象对场景中的物体进行坐标变换和透视投影变换，分别生成左眼视图和右眼视图。实验结果表明，两个目标摄像机与三维模型的位置关系以及基线长度是影响立体效果的重要因素，改变目标摄像机与三维模型的位置，可以分别生成正视差、负视差的立体图像对，当AB与CO的比例参数为0.05时，生成的立体图像对的立体效果较佳。     

A novel multi-object detection method in complex scene using synthetic aperture imaging

This paper proposes a novel multi-object detection method using multiple cameras. Unlike conventional multi-camera object detection methods, our method detects multiple objects using a linear camera array. The array can stream different views of the environment and can be easily reconfigured for a scene compared with the overhead surround configuration. Using the proposed method, the synthesized results can provide not only views of significantly occluded objects but also the ability of focusing on the target while blurring objects that are not of interest. Our method does not need to reconstruct the 3D structure of the scene, can accommodate dynamic background, is able to detect objects at any depth using a new synthetic aperture imaging method based on a simple shift transformation, and can see through occluders. The experimental results show that the proposed method has a good performance and can synthesize objects located within any designated depth interval with much better clarity than that using an existing method. To our best knowledge, it is the first time that such a method using synthetic aperture imaging has been proposed and developed for multi-object detection in a complex scene with a significant occlusion at different depths.