Vision-Based Odometry and SLAM for Medium and High Altitude Flying UAVs

This paper proposes vision-based techniques for localizing an unmanned aerial vehicle (UAV) by means of an on-board camera. Only natural landmarks provided by a feature tracking algorithm will be considered, without the help of visual beacons or landmarks with known positions. First, it is described a monocular visual odometer which could be used as a backup system when the accuracy of GPS is reduced to critical levels. Homography-based techniques are used to compute the UAV relative translation and rotation by means of the images gathered by an onboard camera. The analysis of the problem takes into account the stochastic nature of the estimation and practical implementation issues. The visual odometer is then integrated into a simultaneous localization and mapping (SLAM) scheme in order to reduce the impact of cumulative errors in odometry-based position estimation approaches. Novel prediction and landmark initialization for SLAM in UAVs are presented. The paper is supported by an extensive experimental work where the proposed algorithms have been tested and validated using real UAVs.     

利用影像匹配和摄影测量实现无人机精确导航

GPS技术已广泛应用于无人机的空间定位和导航,但在战时、困难地区的应用会受到限制.本文提出了一种利用影像匹配和摄影测量原理实现无人机精确定位和导航的方法,首先对无人机获取的实时影像和基准影像进行小波变换处理,以获得足够的初选同名点位,然后利用最小二乘影像匹配获取精确的同名点坐标,使用这些同名点,经摄影测量处理得到了精确的无人机空间位置.     

Air‐ground Matching: Appearance‐based GPS‐denied Urban Localization of Micro Aerial Vehicles

In this paper, we address the problem of globally localizing and tracking the pose of a camera‐equipped micro aerial vehicle (MAV) flying in urban streets at low altitudes without GPS. An image‐based global positioning system is introduced to localize the MAV with respect to the surrounding buildings. We propose a novel air‐ground image‐matching algorithm to search the airborne image of the MAV within a ground‐level, geotagged image database. Based on the detected matching image features, we infer the global position of the MAV by back‐projecting the corresponding image points onto a cadastral three‐dimensional city model. Furthermore, we describe an algorithm to track the position of the flying vehicle over several frames and to correct the accumulated drift of the visual odometry whenever a good match is detected between the airborne and the ground‐level images. The proposed approach is tested on a 2 km trajectory with a small quadrocopter flying in the streets of Zurich. Our vision‐based global localization can robustly handle extreme changes in viewpoint, illumination, perceptual aliasing, and over‐season variations, thus outperforming conventional visual place‐recognition approaches. The dataset is made publicly available to the research community. To the best of our knowledge, this is the first work that studies and demonstrates global localization and position tracking of a drone in urban streets with a single onboard camera.     

Photogrammetric processing of high-resolution airborne and satellite linear array stereo images for mapping applications

This article introduces a mathematical model for photogrammetric processing of linear array stereo images acquired by high-resolution satellite imaging systems such as IKONOS. The experimental result of the generation of simulated IKONOS stereo images based on photogrammetric principles, IKONOS imaging geometry and a set of georeferenced aerial images is presented. An accuracy analysis of ground points derived from the simulated IKONOS stereo images is performed. The impact of the number of GCPs (ground control points), distribution of GCPs, and image measurement errors on the ground point accuracy is investigated. It is concluded that an accuracy of ground coordinates from 2 m to 3 m is attainable with GCPs, and 5 m to 12 m without GCPs. Two data sets of HRSC (high resolution stereo camera) and MOMS (modular opto-electronic multispectral stereo-scanner)-2P are also utilized to test the model and system. The presented data processing method is a key to the generation of mapping products such as digital terrain models (DEM) and digitial shorelines from high-resolution satellite images.     

Navigation Aided Image Processing in UAV Surveillance: Preliminary Results and Design of an Airborne Experimental System

This paper describes an airborne reconfigurable measurement system being developed at Swedish Defence Research Agency (FOI), Sensor Technology, Sweden. An image processing oriented sensor management architecture for UAV (unmanned aerial vehicles) IR/EO-surveillance is presented. Some preliminary results of navigation aided image processing in UAV applications are demonstrated, such as SLAM (simultaneous localization and mapping), structure from motion and geolocation, target tracking, and detection of moving objects. The design goal of the measurement system is to emulate a UAV-mounted sensor gimbal using a stand-alone system. The minimal configuration of the system consists of a gyro-stabilized gimbal with IR and CCD sensors and an integrated high-performance navigation system. The navigation system combines dGPS real-time kinematics (RTK) data with data from an inertial measurement unit (IMU) mounted with reference to the optical sensors. The gimbal is to be used as an experimental georeferenced sensor platform, using a choice of carriers, to produce military relevant image sequences for studies of image processing and sensor control on moving surveillance and reconnaissance platforms. Furthermore, a high resolution synthetic environment, developed for sensor simulations in the visual and infrared wavelengths, is presented. © 2004 Wiley Periodicals, Inc.     

Embedded omni-vision navigator based on multi-object tracking

This paper presents an embedded omni-vision navigation system which involves landmark recognition, multi-object tracking, and vehicle localization. A new tracking algorithm, the feature matching embedded particle filter, is proposed. Landmark recognition is used to provide the front-end targets. A global localization method for omni-vision based on coordinate transformation is also proposed. The digital signal processor (DSP) provides a hardware platform for on-board tracker. Dynamic navigator employs DSP tracker to follow the landmarks in real time during the arbitrary movement of the vehicle and computes the position for localization based on time sequence images analysis. Experimental results demonstrated that the navigator can efficiently offer the vehicle guidance.     

低空无人机航空摄影高度自动测量方法研究

为控制低空无人机摄影高度,获得更加清晰的地理信息图像,需要对低空无人机摄影高度自动测量方法进行优化研究;当前方法主要利用射影几何知识的自动化标定方法实现低空无人机航空摄影高度的自动测量;该方法存在噪声影响严重,且测量误差较大的问题;为此,提出一种基于多传感器与卡尔曼滤波相结合的低空无人机航空摄影高度自动测量方法;该方法首先通过分析气压测量法计算各种气压因素对低空无人机航空摄影高度的影响,然后推导出大气对流层内气压随低空无人机航空摄影高度的变化;然后采用双GPS系统同时工作,对GPS、气压高度计和IMU测量获得的低空无人机航空摄影高度信号进行冗余备份;采用基于二阶多项式的修正方法对低空无人机航空摄影传感器输出值进行补偿和修正;根据动力学方程建立低空无人机航空摄影的动力学方程获得高度测量状态方程;最后采用卡尔曼滤波的线性最小方差估计准则对低空无人机航空摄影高度进行均方差估计计算,实现低空高度自动测量与校正。实验结果表明,所提方法具有精度高、收敛性好且滤波效果理想的优势。     

Vision-aided Estimation of Attitude,Velocity, and Inertial Measurement Bias for UAV Stabilization

This paper studies vision-aided inertial navigation of small-scale unmanned aerial vehicles (UAVs) in GPS-denied environments. The objectives of the navigation system are to firstly online estimate and compensate the unknown inertial measurement biases, secondly provide drift-free velocity and attitude estimates which are crucial for UAV stabilization control, and thirdly give relatively accurate position estimation such that the UAV is able to perform at least a short-term navigation when the GPS signal is not available. For the vision system, we do not presume maps or landmarks of the environment. The vision system should be able to work robustly even given low-resolution images (e.g., 160 ×120 pixels) of near homogeneous visual features. To achieve these objectives, we propose a novel homography-based vision-aided navigation system that adopts four common sensors: a low-cost inertial measurement unit, a downward-looking monocular camera, a barometer, and a compass. The measurements of the sensors are fused by an extended Kalman filter. Based on both analytical and numerical observability analyses of the navigation system, we theoretically verify that the proposed navigation system is able to achieve the navigation objectives. We also show comprehensive simulation and real flight experimental results to verify the effectiveness and robustness of the proposed navigation system.     

Development of Unmanned Aerial Vehicle (UAV) system with waypoint tracking and vision-based reconnaissance

This paper introduces the developed UAV system for low cost operation and an EOS (Electro Optical System) laboratory. This paper highlights an autonomous navigation system based on microcontrollers that can track a target using images, take three-dimensional measurements of the target, and acquire high quality images. The hardware system and an algorithm for the EOS verify the performance of the image tracking system and 3-D measurement of the target’s position. 3-D position estimations for the target are solved using the mathematical relationship between the UAV and target. Although an on-board EOS can make errors in 3-D measurement, the proposed approach shows improved accuracy and confidence for 3-D target tracking using a postprocessing method.     

A Vision-Based Guidance System for UAV Navigation and Safe Landing using Natural Landmarks

In this paper a vision-based approach for guidance and safe landing of an Unmanned Aerial Vehicle (UAV) is proposed. The UAV is required to navigate from an initial to a final position in a partially known environment. The guidance system allows a remote user to define target areas from a high resolution aerial or satellite image to determine either the waypoints of the navigation trajectory or the landing area. A feature-based image-matching algorithm finds the natural landmarks and gives feedbacks to an onboard, hierarchical, behaviour-based control system for autonomous navigation and landing. Two algorithms for safe landing area detection are also proposed, based on a feature optical flow analysis. The main novelty is in the vision-based architecture, extensively tested on a helicopter, which, in particular, does not require any artificial landmark (e.g., helipad). Results show the appropriateness of the vision-based approach, which is robust to occlusions and light variations.     

Visual 3-D SLAM from UAVs

The aim of the paper is to present, test and discuss the implementation of Visual SLAM techniques to images taken from Unmanned Aerial Vehicles (UAVs) outdoors, in partially structured environments. Every issue of the whole process is discussed in order to obtain more accurate localization and mapping from UAVs flights. Firstly, the issues related to the visual features of objects in the scene, their distance to the UAV, and the related image acquisition system and their calibration are evaluated for improving the whole process. Other important, considered issues are related to the image processing techniques, such as interest point detection, the matching procedure and the scaling factor. The whole system has been tested using the COLIBRI mini UAV in partially structured environments. The results that have been obtained for localization, tested against the GPS information of the flights, show that Visual SLAM delivers reliable localization and mapping that makes it suitable for some outdoors applications when flying UAVs.     

Detection of collapsed buildings with the aerial images captured from UAV基于低空无人机航拍图像的坍塌建筑物自动探测方法

In this paper, we present a method of detecting the collapsed buildings with the aerial images which are captured by an unmanned aerial vehicle (UAV) for the postseismic evaluation. Different from the conventional methods that apply the satellite images or the high-altitude UAV for the coarse disaster evaluation over large area, the purpose of this work is to achieve the accurate detection of collapsed buildings in small area from low altitude. By combining the motion and appearance features of collapsed buildings extracted from successive aerial images, each pixel in the input image will be measured by a statistical method where the background pixels will be penalized and the ones of collapsed buildings will be assigned with high value. The candidates of collapsed buildings will be established by integrating the extracted feature points into local groups with the online clustering algorithm. To reduce the false alarm caused by the complex background noise, each predicted candidate will be further verified by the temporal tracking framework where both the trajectory and the appearance of a candidate will be measured. The candidate of collapsed buildings that can survive through long time will be considered as true positive, otherwise rejected as a false alarm. Through extensive experiments, the efficiency and the effectiveness of proposed algorithm have been proved.     

无人机遥感影像镶嵌技术综述

无人机影像镶嵌是低空遥感数据处理系统中的一个重要内容,其目的是将序列无人机影像拼接成一幅具有地理坐标的影像。为了使该领域的研究人员对当前无人机影像镶嵌方法有较全面的了解,对各种无人机影像镶嵌方法进行了综述。多项式法、卡尔曼滤波法、基于SfM点云匹配和传统空中三角测量法均需要地面控制点,而对偶四元数POS辅助空中三角测量方法无需或者只需少量地面控制点,将在无人机影像镶嵌领域有较为广阔的应用前景。     

利用地面景象信息辅助的无人机自主定位技术

针对无人机自主定位过程中GPS定位系统失效的问题,提出了一种利用地面景象信息辅助的无人机自主定位技术,首先利用无人机所拍摄的实时航拍图像,与预先储存在无人机计算机中地面景象的数字化地形图进行匹配,从匹配结果中提取一个同名像点,结合地面景象数字化地形图所提供的数据信息获取此同名像点的地理位置坐标,根据同名像点位置与无人机位置间的几何关系,结合机载光电测量系统的坐标转换过程,实现无人机的自主定位过程。利用已知的地面同名像点的地理位置信息,反推出无人机的地理位置信息具有一定的创新性。由于整个定位过程存在实际误差,因此利用无人机飞行时记录的数据,采用蒙特卡罗法对定位误差进行仿真试验。试验结果表明该技术能够在误差允许范围内,在GPS定位系统失效的情况下完成无人机的自主定位     

Vision-based navigation of unmanned aerial vehicles

This paper presents a vision-based navigation strategy for a vertical take-off and landing (VTOL) unmanned aerial vehicle (UAV) using a single embedded camera observing natural landmarks. In the proposed approach, images of the environment are first sampled, stored and organized as a set of ordered key images (visual path) which provides a visual memory of the environment. The robot navigation task is then defined as a concatenation of visual path subsets (called visual route) linking the current observed image and a target image belonging to the visual memory. The UAV is controlled to reach each image of the visual route using a vision-based control law adapted to its dynamic model and without explicitly planning any trajectory. This framework is largely substantiated by experiments with an X4-flyer equipped with a fisheye camera.     

Stable camera position control of unmanned aerial vehicle with three‐degree‐of‐freedom manipulator for visual test of bridge inspection

This paper describes a camera position control with aerial manipulator for visual test of bridge inspection. Our developed unmanned aerial vehicle (UAV) has three‐degree‐of‐freedom (3‐DoF) manipulator on its top to execute visual or hammering test of the inspection. This paper focuses on the visual test. A camera was implemented at the end of the manipulator to acquire images of the narrow space of the bridge such as bearings, which the conventional UAV without the camera‐attached manipulators at its top cannot achieve the fine visual test. For the visual test, it is desirable that the camera is above the body with enough distance between the camera and the body. As obvious, the camera position in the inertial coordinate system is effected by the movement of the body. Therefore we implement the camera position control compensating the body movement into the UAV. As a result of an experiment, it is assessed that the proposed control reduces the position error of the camera comparing the one of the body. The mean position error of the camera is 0.039 m that is 51.4% of the one of the body. Our world‐first study enables to acquire the image of the bearing of the bridge by a camera mounted at the end effector of aerial manipulator fixed on UAV.     

Low‐altitude Terrestrial Spectroscopy from a Pushbroom Sensor

Hyperspectral cameras sample many different spectral bands at each pixel, enabling advanced detection and classification algorithms. However, their limited spatial resolution and the need to measure the camera motion to create hyperspectral images makes them unsuitable for nonsmooth moving platforms such as unmanned aerial vehicles (UAVs). We present a procedure to build hyperspectral images from line sensor data without camera motion information or extraneous sensors. Our approach relies on an accompanying conventional camera to exploit the homographies between images for mosaic construction. We provide experimental results from a low‐altitude UAV, achieving high‐resolution spectroscopy with our system.     

基于多视角多监督网络的无人机图像定位方法

针对现有跨视角图像匹配算法精度低的问题,提出了一种基于多视角多监督网络（MMNet）的无人机（UAV）定位方法。首先,所提方法融合卫星视角和UAV视角,在统一的网络架构下学习全局和局部特征并以多监督方式训练分类网络并执行度量任务。具体来说,MMNet主要采用了重加权正则化三元组损失（RRT）学习全局特征,该损失利用重加权和距离正则化加权策略来解决多视角样本不平衡以及特征空间结构紊乱的问题。同时,为了关注目标地点中心建筑的上下文信息,MMNet对特征图进行方形环切割来获取局部特征。然后,分别用交叉熵损失和RRT执行分类和度量任务。最终,使用加权策略聚合全局和局部特征来表征目标地点图像。通过在当前流行的UAV数据集University-1652上进行实验,可知MMNet在UAV定位任务的召回率Recall@1 (R@1)及平均精准率（AP）上分别达到83.97%和86.96%。实验结果表明,相较于LCM、SFPN等方法,MMNet显著提升了跨视角图像的匹配精度,进而增强了UAV图像定位的实用性。     

Image Based and Hybrid Visual Servo Control of an Unmanned Aerial Vehicle

The use of unmanned aerial vehicles (UAVs) for military, scientific, and civilian sectors are increasing drastically in recent years. This study presents algorithms for the visual-servo control of an UAV, in which a quadrotor helicopter has been stabilized with visual information through the control loop. Unlike previous study that use pose estimation approach which is time consuming and subject to various errors, the visual-servo control is more reliable and fast. The method requires a camera on-board the vehicle, which is already available on various UAV systems. The UAV with a camera behaves like an eye-in-hand visual servoing system. In this study the controller was designed by using two different approaches; image based visual servo control method and hybrid visual servo control method. Various simulations are developed on Matlab, in which the quadrotor aerial vehicle has been visual-servo controlled. In order to show the effectiveness of the algorithms, experiments were performed on a model quadrotor UAV, which suggest successful performance.