Novel calibration method for camera array in spherical arrangement

A camera array placed in a spherical arrangement facilitates a new paradigm for future interactive visual applications, and the calibration of the cameras is a crucial task for such implementation. It is difficult to apply previous calibration methods to this special camera system because opposite features can commonly be found among the opposing cameras used in this system. In this case, a part-by-part calibration may fail in terms of error accumulation. In this paper, we propose an opposite feature based camera calibration method for a camera array in a spherical arrangement. An adaptive scheme is designed on a cube with opposite colour patterns and optimisations in handling the above problem of opposite features being invisible to cameras at opposite positions, based upon which, with our method, a common visible feature point is not required in advance. By applying a practical camera array in a spherical arrangement, and using 3D modelling, the superiority of the proposed method was verified through computer simulations. The results demonstrate that the proposed calibration method can contribute to the imaging system and future visual applications.     

双目立体视觉中靶标的设计与识别

在进行实验室自主研制的双目主动立体视觉监测平台的研究过程中,需要预先知道两个摄像机的初始位置,为了完成初始位置的标定,我们设计了特殊的靶标。将靶标均匀设置在圆形导轨内侧,利用数码摄像机作为图像传感器,通过对靶标上的编码标志点进行识别和检测,确定靶标的编号;且利用非编码标志点与编码标志点的相对位置关系,通过角点检测等步骤实现对非编码标志点的识别,进而完成对整个靶标的识别。     

Distributed target localization using a progressive certainty map in visual sensor networks

Collaboration in visual sensor networks (VSNs) is essential not only to compensate for the processing, sensing, energy, and bandwidth limitations of each sensor node but also to improve the accuracy and robustness of the network. In this paper, we study target localization in VSNs, a challenging computer vision problem because of two unique features of cameras, including the extremely higher data rate and the directional sensing characteristics with limited field of view. Traditionally, the problem is solved by localizing the targets at the intersections of the back-projected 2D cones of each target. However, the existence of visual occlusion among targets would generate many false alarms. In this work, instead of resolving the uncertainty about target existence at the intersections, we identify and study the non-occupied areas in the cone and generate the so-called certainty map of non-existence of targets. As a result, after fusing inputs from a set of sensor nodes, the unresolved regions on the certainty map would be the location of targets. This paper focuses on the design of a light-weight, energy-efficient, and robust solution where not only each camera node transmits a very limited amount of data but that a limited number of camera nodes is involved. We propose a dynamic itinerary for certainty map integration where the entire map is progressively clarified from sensor to sensor. When the confidence of the certainty map is satisfied, targets are localized at the remaining unresolved regions in the certainty map. Based on results obtained from both simulation and real experiments, the proposed progressive method shows effectiveness in detection accuracy as well as energy and bandwidth efficiency.     

Non-overlapping Distributed Tracking System Utilizing Particle Filter

Tracking people across multiple cameras is a challenging research area in visual computing, especially when these cameras have non-overlapping field of views. The important task is to associate a current subject with other prior appearances of the same subject across time and space in a camera network. Several known techniques rely on Bayesian approaches to perform the matching task. However, these approaches do not scale well when the dimension of the problem increases; e.g. when the number of subject or possible path increases. The aim of this paper is to propose a unified tracking framework using particle filters to efficiently switch between visual tracking (field of view tracking) and track prediction (non-overlapping region tracking). The particle filter tracking system utilizes a map (known environment) to assist the tracking process when targets leave the field of view of any camera. We implemented and tested this tracking approach in an in-house multiple cameras system as well as using on-line data. Promising results were obtained which suggested the feasibility of such an approach.

A second-order conic optimization-based method for visual servoing

This work presents a novel method for the visual servoing control problem based on second-order conic optimization. Special cases of the proposed method provide similar results as those obtained by the position-based and image-based visual servoing methods. The goal in our approach is to minimize both the end-effector trajectory in the Cartesian space and image feature trajectories simultaneously. For this purpose, a series of second-order conic optimization problems is solved. Each problem starts from the current camera pose and finds the camera velocity as well as the next camera pose such that (1) the next camera pose is as close as possible to the line connecting the initial and desired camera poses, and (2) the next feature points are as close as possible to the corresponding lines connecting the initial and desired feature points. To validate our approach, we provide simulations and experimental results for several different camera configurations.     

A hardware in the loop simulation testbed for vision-based leader-follower formation flight

This paper proposes a hardware in the loop simulation testbed by using airborne seekers (an onboard visual tracking system) as a relative measurement sensor in the leader-follower formation flight. To the best of the authors’ knowledge, this is the first study on experimental applications of airborne seekers in unmanned aircraft formation flight. The proposed structure gives an effective method to study the effects of uncertainties such as the camera process delay and the seeker measurement noises on formation keeping. Compensation of errors in visual measurements is considered as well.     

Eye-to-hand approach on eye-in-hand configuration within real-time visual servoing

This paper presents a framework of hand-eye relation for visual servoing with more precision, mobility, and global view. Mainly, there are two types of camera utilization for visual servoing: eye-in-hand and eye-to-hand configurations. Both have own merits and drawbacks regarding to precision and view limit that oppose each other. Based on both behaviors, this paper employs a mobile manipulator as the second robot to hold the camera. Here, the camera architecture is eye-to-hand configuration for the main robot, but mainly behaves as eye-in-hand configuration for the second robot. Having this framework, the drawback of each configuration is resolved by the benefit of the other. Here, the camera becomes mobile with more precision and global view. In addition, since there is no additional camera, the vision algorithm can be kept simple. In order to gain real-time visual servoing, this paper also addresses real-time constraints on vision system and data communication between robot and vision. Here, a hexagon pattern of artificial marker with a simplified image processing is developed. A grasp positioning problem is considered with position-based dynamic look and move visual control through object pose estimation. The system performance is validated by the experimental result.     

A real-time multitarget tracking system with robust multichannel CNN-UM algorithms

This paper introduces a tightly coupled topographic sensor-processor and digital signal processor (DSP) architecture for real-time visual multitarget tracking (MTT) applications. We define real-time visual MTT as the task of tracking targets contained in an input image flow at a sampling-rate that is higher than the speed of the fastest maneuvers that the targets make. We utilize a sensor-processor based on the cellular neural network universal machine architecture that permits the offloading of the main image processing tasks from the DSP and introduces opportunities for sensor adaptation based on the tracking performance feedback from the DSP. To achieve robustness, the image processing algorithms running on the sensor borrow ideas from biological systems: the input is processed in different parallel channels (spatial, spatio-temporal and temporal) and the interaction of these channels generates the measurements for the digital tracking algorithms. These algorithms (running on the DSP) are responsible for distance calculation, state estimation, data association and track maintenance. The performance of the proposed system is studied using actual hardware for different video flows containing rapidly moving maneuvering targets.     

基于IMU/视觉融合的导航定位算法研究

针对基于视觉传感器的移动机器人在快速运动或发生旋转时出现图像模糊和特征丢失,以至无法进行特征匹配,从而导致系统定位和建图的准确度及精确度下降问题,该文提出了一种以深度相机(RGB_D)融合惯性测量单元（IMU）的方案。采用ORB SLAM2算法进行位姿估计,同时将IMU信息作为约束弥补相机数据的缺失。两种传感器的测量数据采用基于扩展卡尔曼滤波的松耦合方式进行非线性优化,通过数据采集实验表明,该方法能有效提高机器人的定位精度和系统建图效果。     

一种视觉SLAM单目半稠密建图方法的实现

SLAM即同时定位与地图构建,一直是机器人和计算机视觉的研究热点。尤其是视觉SLAM技术,21世纪以来在理论和实践上均取得了明显的突破,已逐步迈向市场应用。建图作为SLAM的两大目标之一,可以满足更多的应用需求。本文在给定相机轨迹的情况下,提出一种视觉SLAM单目半稠密建图方法,利用极线搜索和块匹配技术,加入图像变换和逆深度高斯深度滤波器处理,以期避免单目稠密建图严重依赖纹理、计算量大的缺点,提高单目半稠密建图的准确性和鲁棒性。经测试显示,改进的单目半稠密建图方法在检测梯度变化明显像素点上更加准确,深度估计的平均误差和平方误差分别减少了9%和47%,是一种可行有效的视觉SLAM单目半稠密建图解决方案。     

A vision-based system for robotic inspection of marine vessels

This paper presents a novel intelligent system for the automatic visual inspection of vessels consisting of three processing levels: (a) data acquisition: images are collected using a magnetic climbing robot equipped with a low-cost monocular camera for hull inspection; (b) feature extraction: all the images are characterized by 12 features consisting of color moments in each channel of the HSV space; (c) classification: a novel tool, based on an ensemble of classifiers, is proposed to classify sub-images as rust or non-rust. This paper provides a helpful roadmap to guide future research on the detection of rusting of metals using image processing.     

SCARA机器人实验平台视觉标定研究

文中针对eye-to-hand视觉标定中机械手的移动可能对目标造成遮挡的问题,利用eye-in-hand摄像机和eye-to-hand摄像机结合的方式,以SCARA工业机器人为研究对象,建立双摄像机视觉标定系统。同时,文中提出了一种简单有效的eye-in-hand的标定方式,以机器人末端坐标系为辅助坐标系,建立基坐标到辅助坐标再到摄像坐标的对应关系,从而为实验室平台下工业机器人的目标定位提供了一种有效的方式。     

智能视频监控系统中多摄像头协同跟踪算法研究

为了提高智能视频监控系统中运动目标跟踪的准确率和有效性,本文利用数据融合和协同跟踪技术,实现了多摄像头协同跟踪系统中运动目标的有效跟踪。首先,利用单摄像头独自跟踪得到目标图像,然后利用平面单应性和极线几何约束结合实现多摄像头视图间的目标匹配和协同跟踪,最后得到精确的目标跟踪结果。通过在MATLAB上实验仿真,并与单摄像头目标跟踪进行对比,得出该算法具有较好的跟踪效果以及抗遮挡性能,可广泛应用于智能视频监控系统中。     

Using genetic algorithms and neural networks for surface land minedetection

Knowledge based techniques have been used to automatically detect surface land mines present in thermal and multispectral images. Polarization-sensitive infrared sensing is used to highlight the polarization signature of man-made targets such as land mines over natural features in the image. Processing the thermal polarization images using a background-discrimination algorithm, we were able to successfully identify eight of the nine man-made targets, three of which were mines, with only three false targets. A digital camera is used to collect a number of multispectral bands of the test mine area containing three surface land mines with natural and man-made clutter. Using a supervised and unsupervised neural network technique on the textural and spectral characteristics of selected multispectral bands (using a genetic algorithm tool), we successfully identified the three surface mines but obtained numerous false targets with varying degrees of accuracy. Finally, to further improve our detection of land mines, we use a fuzzy rule-based fusion technique on the processed polarization resolved image together with the output results of the two best classifiers. Fuzzy rule-based fusion identified the locations of all three land mines and reduced the number of false alarms from seven (as obtained by the polarization resolved image) to two. Additional experiments on several other images have also produced favorable results at this early stage in testing the algorithm and comparing it with an existing commercial system     

基于像素偏振片阵列的实时水下目标成像技术

借助偏振成像可以增强水下目标的探测效果。传统的偏振成像方法需要光学检偏器的机械转动来实现,这限制了其在水下的实时探测性能。采用基于像素偏振片阵列图像传感器开发的相机设计了一种水下实时成像系统。系统通过阵列上排布的四向微偏振片一次性捕获四向偏振图像,从而全局估算背景杂散光的偏振角和偏振度。然后利用偏振信息反解得到杂散光光强,最后借助水下成像物理模型得到去散射后的目标增强图像。实验结果表明,将像素偏振片阵列图像传感器应用到水下成像能够有效增强水下图像的对比度,且成像处理过程实时快速,进一步提高了水下目标的探测效率。     

Recognition of dynamic video contents with global probabilistic models of visual motion.

The exploitation of video data requires methods able to extract high-level information from the images. Video summarization, video retrieval, or video surveillance are examples of applications. In this paper, we tackle the challenging problem of recognizing dynamic video contents from low-level motion features. We adopt a statistical approach involving modeling, (supervised) learning, and classification issues. Because of the diversity of video content (even for a given class of events), we have to design appropriate models of visual motion and learn them from videos. We have defined original parsimonious global probabilistic motion models, both for the dominant image motion (assumed to be due to the camera motion) and the residual image motion (related to scene motion). Motion measurements include affine motion models to capture the camera motion and low-level local motion features to account for scene motion. Motion learning and recognition are solved using maximum likelihood criteria. To validate the interest of the proposed motion modeling and recognition framework, we report dynamic content recognition results on sports videos.     

Positioning beacon system using digital camera and LEDs

This paper is on a novel use of lighting or signaling devices constructed by light-emitting diodes (LEDs) as a positioning beacon. The idea is that the surface of the LED lighting device is divided into regions and used to show different visual patterns that are not noticeable by the human eye due to the high-frequency switching of the LEDs. A digital camera is used as a receiver to capture a sequence of images of the LED positioning beacon transmitter. Image-processing algorithms are used to decode the location code that is encoded in the visual patterns transmitted by the LED device. This idea can be applied to any LED traffic lights or signaling devices on the road and turn them into parts of a positioning beacon system. Such a system made up of high-brightness visible LEDs can provide the function of open-space wireless broadcasting of the positioning signal. The LED signaling method, transmission protocol, camera frame rate, LED flash rate, together with an implemented system and the experimental results, are presented in this paper.     

基于小尺寸靶标组合的大视场摄像机标定方法

摄像机标定是计算机视觉中的一个重要过程。大视 场摄像机标定中,大尺寸靶标加工困难,而使 用单一小靶标只能得到局部最优解。本文针对这一问题,提出一种基于小尺寸靶标组合的 大视场摄像机标定方法。利用摄像机多项式投影模型,通过差分式求解方法将各个小尺寸靶 标组合成一个大尺寸靶标,然后使 用大尺寸靶标对大视场摄像机进行精确标定;在小尺寸靶标组合过程中,为了消除 利用差分式方 法进行求解时各个小尺寸靶标间只能平移、不能旋转的安放限制,提出了一种基于交比不变 性的直线夹角 求解方法,并设计了相应的复合靶标,提高了方法的灵活性。仿真和实际的实验结果表明, 本文所提方法能够实现对大视场摄像机的精确标定。     

Efficient visual object detection with spatially global Gaussian mixture models and uncertainties

In this paper, we deal with the problem of visual detection of moving objects using innovative Gaussian mixture models (GMM). The proposed method, the Spatially Global Gaussian Mixture Model (SGGMM) uses RGB and pixel uncertainty for background modelling. The SGGMM with colours only is used for scenes with moderate illumination changes. When combined with pixel uncertainty statistics, the method can deal efficiently with dynamic backgrounds and scene backgrounds with fast change in luminosity. Experimental evaluation in indoor and outdoor environments shows the performance of the foreground segmentation with the proposed SGGMM model using solely RGB colour or in combination with pixel uncertainties. These experimental scenarios take into account changes in the background within the scene. They are also used to compare the proposed technique with other state-of-the-art segmentation approaches in terms of accuracy and execution time performance. Further, our solution is implemented and tested in embedded camera sensor network nodes.     

TapTell: Interactive visual search for mobile task recommendation

Mobile devices are becoming ubiquitous. People use them as personal concierge to search information and make decisions. Therefore, understanding user intent and subsequently provide meaningful and personalized suggestions is important. While existing efforts have predominantly focused on understanding the intent expressed by a textual or a voice query, this paper presents a new and alternative perspective which understands user intent visually, i.e., via visual signal captured by the built-in camera. We call this kind of intent “visual intent” as it can be naturally expressed through a visual form. To accomplish the discovery of visual intent on the phone, we develop TapTell, an exemplary real application on Windows Phone seven, by taking advantages of user interaction and rich context to enable interactive visual searches and contextual recommendations. Through the TapTell system, a mobile user can take a photo and indicate an object-of-interest within the photo via different drawing patterns. Then, the system performs a search-based recognition using a proposed large-scale context-embedded vocabulary tree. Finally, contextually relevant entities (i.e., local businesses) are recommended to the user for completing mobile tasks (those tasks which are natural to be raised and subsequently executed when the user utilizes mobile devices). We evaluated TapTell in a variety of scenarios with millions of images, and compared our results to state-of-the-art algorithms for image retrieval.