Vision-Based Odometry and SLAM for Medium and High Altitude Flying UAVs

This paper proposes vision-based techniques for localizing an unmanned aerial vehicle (UAV) by means of an on-board camera. Only natural landmarks provided by a feature tracking algorithm will be considered, without the help of visual beacons or landmarks with known positions. First, it is described a monocular visual odometer which could be used as a backup system when the accuracy of GPS is reduced to critical levels. Homography-based techniques are used to compute the UAV relative translation and rotation by means of the images gathered by an onboard camera. The analysis of the problem takes into account the stochastic nature of the estimation and practical implementation issues. The visual odometer is then integrated into a simultaneous localization and mapping (SLAM) scheme in order to reduce the impact of cumulative errors in odometry-based position estimation approaches. Novel prediction and landmark initialization for SLAM in UAVs are presented. The paper is supported by an extensive experimental work where the proposed algorithms have been tested and validated using real UAVs.     

SOFT‐SLAM: Computationally efficient stereo visual simultaneous localization and mapping for autonomous unmanned aerial vehicles

Autonomous navigation of unmanned aerial vehicles (UAVs) in GPS‐denied environments is a challenging problem, especially for small‐scale UAVs characterized by a small payload and limited battery autonomy. A possible solution to the aforementioned problem is vision‐based simultaneous localization and mapping (SLAM), since cameras, due to their dimensions, low weight, availability, and large information bandwidth, circumvent all the constraints of UAVs. In this paper, we propose a stereo vision SLAM yielding very accurate localization and a dense map of the environment developed with the aim to compete in the European Robotics Challenges (EuRoC) targeting airborne inspection of industrial facilities with small‐scale UAVs. The proposed approach consists of a novel stereo odometry algorithm relying on feature tracking (SOFT), which currently ranks first among all stereo methods on the KITTI dataset. Relying on SOFT for pose estimation, we build a feature‐based pose graph SLAM solution, which we dub SOFT‐SLAM. SOFT‐SLAM has a completely separate odometry and mapping threads supporting large loop‐closing and global consistency. It also achieves a constant‐time execution rate of 20 Hz with deterministic results using only two threads of an onboard computer used in the challenge. The UAV running our SLAM algorithm obtained the highest localization score in the EuRoC Challenge 3, Stage IIa–Benchmarking, Task 2. Furthermore, we also present an exhaustive evaluation of SOFT‐SLAM on two popular public datasets, and we compare it to other state‐of‐the‐art approaches, namely ORB‐SLAM2 and LSD‐SLAM. The results show that SOFT‐SLAM obtains better localization accuracy on the majority of datasets sequences, while also having a lower runtime.     

Bearing-only visual SLAM for small unmanned aerial vehicles in GPS-denied environments

This paper presents a hierarchical simultaneous localization and mapping(SLAM) system for a small unmanned aerial vehicle(UAV) using the output of an inertial measurement unit(IMU) and the bearing-only observations from an onboard monocular camera.A homography based approach is used to calculate the motion of the vehicle in 6 degrees of freedom by image feature match.This visual measurement is fused with the inertial outputs by an indirect extended Kalman filter(EKF) for attitude and velocity estimation.Then,another EKF is employed to estimate the position of the vehicle and the locations of the features in the map.Both simulations and experiments are carried out to test the performance of the proposed system.The result of the comparison with the referential global positioning system/inertial navigation system(GPS/INS) navigation indicates that the proposed SLAM can provide reliable and stable state estimation for small UAVs in GPS-denied environments.     

无人机视觉SLAM环境感知发展研究

无人机在进行搜索救援等高级任务的时候,往往需要确定自己的位置和环境信息;仿照于人类通过视觉感知环境信息,视觉SLAM是计算机视觉领域里面通过视觉传感器感知环境的信息并快速跟踪自身的位置和建立环境地图的一种前沿技术;文章首先阐述了 VSLAM的重要组成部分:前端处理(特征点法和直接法)、数据关联、后端优化算法(滤波方法和优化方法)和建图;然后总结了一些在无人机上成功应用的典型VSLAM算法,以及在VSLAM发展的30多年的时间里涌现出许多出色的方案和研究机构;接着论述了当前用于无人机VSLAM发展的几个重点问题,多无人机协同的C—SLAM、深度学习和语义分割在SLAM中的应用、以视觉惯导为代表的多传感器融合SLAM;最后,对VSLAM方法进行总结,给出了未来的发展方向,希望对后续研究提供指导和帮助.     

Architectures for Cooperative Airborne Simultaneous Localisation and Mapping

This paper develops active Simultaneous Localisation And Mapping (SLAM) trajectory control strategies for multiple cooperating Unmanned Aerial Vehicles (UAVs) for tasks such as surveillance and picture compilation in Global Positioning System (GPS)-denied environments. Each UAV in the team uses inertial sensor and terrain sensor information to simultaneously localise the UAV while building a point feature map of the surrounding terrain, where map information is shared between vehicles over a data fusion network. Multi-vehicle active SLAM control architectures are proposed that actively plan the trajectories and motions of each of the vehicles in the team based on maximising information in the localisation and mapping estimates. We demonstrate and compare an ideal, centralised architecture, where a central planning node chooses optimal actions for each UAV, and a coordinated, decentralised architecture, where UAVs make their own control decisions based on common shared map information. The different architectures involve varying degrees of complexity and optimality through differing communications and computational requirements. Results are presented using a three-UAV team in a six-degree of freedom multi-UAV simulator.     

基于在线光度标定的半直接视觉SLAM算法

为解决视觉 SLAM（同时定位与地图创建）算法依赖图像亮度而对光照变化场景敏感的问题,提出一种基于在线光度标定的半直接视觉 SLAM 算法。首先,根据相机成像原理,提出基于光度标定的帧间位姿估计方法,在求解位姿的同时对原始的输入图像进行光度校正。其次,在特征追踪环节采取最近共视关键帧匹配策略,以提升特征点匹配效率。最后,对后端重投影迭代优化策略进行改进,降低光照变化对视觉 SLAM 算法的精度和鲁棒性的影响。在 TUM、EuRoC 数据集上的实验结果表明,本算法的轨迹估计精度优于 LSD-SLAM 和 SVO 2.0 算法,尤其是在中等难度、高难度的数据集序列上。在真实环境测试中,通过对比本算法与激光方法的轨迹估计结果,证明本算法有效提高了传统视觉 SLAM 方法在光照不均匀场景下的定位精度与鲁棒性。     

Computer Vision Onboard UAVs for Civilian Tasks

Computer vision is much more than a technique to sense and recover environmental information from an UAV. It should play a main role regarding UAVs’ functionality because of the big amount of information that can be extracted, its possible uses and applications, and its natural connection to human driven tasks, taking into account that vision is our main interface to world understanding. Our current research’s focus lays on the development of techniques that allow UAVs to maneuver in spaces using visual information as their main input source. This task involves the creation of techniques that allow an UAV to maneuver towards features of interest whenever a GPS signal is not reliable or sufficient, e.g. when signal dropouts occur (which usually happens in urban areas, when flying through terrestrial urban canyons or when operating on remote planetary bodies), or when tracking or inspecting visual targets—including moving ones—without knowing their exact UMT coordinates. This paper also investigates visual servoing control techniques that use velocity and position of suitable image features to compute the references for flight control. This paper aims to give a global view of the main aspects related to the research field of computer vision for UAVs, clustered in four main active research lines: visual servoing and control, stereo-based visual navigation, image processing algorithms for detection and tracking, and visual SLAM. Finally, the results of applying these techniques in several applications are presented and discussed: this study will encompass power line inspection, mobile target tracking, stereo distance estimation, mapping and positioning.     

UAV Localization in Row Crops

High‐flying unmanned aerial vehicles (UAVs) are transforming industrial and research agriculture by delivering high spatiotemporal resolution data on a field environment. While current UAVs fly high above fields collecting aerial imagery, future low‐flying aircraft will directly interact with the environment and will utilize a wider variety of sensors. Safely and reliably operating close to unstructured environments requires improving UAVs' sensing, localization, and control algorithms. To this end, we investigate localizing a micro‐UAV in corn phenotyping trials using a laser scanner and IMU to control the altitude and position of the vehicle relative to the plant rows. In this process, the laser scanner is not only a means of localization, but also a scientific instrument for measuring plant properties. Experimental evaluations demonstrate that the is capable of safely and reliably operating in real‐world phenotyping trials. We experimentally validate the system in both low and high wind conditions in fully mature corn fields. Using test data from 18 test flights, we show that the UAV is capable of localizing its position to within one field row of the true position.     

GMSK-SLAM: a new RGB-D SLAM method with dynamic areas detection towards dynamic environments

As a research hotspot in the field of robotics, Simultaneous localization and mapping (SLAM) has made great progress in recent years, but few SLAM algorithms take dynamic or movable targets in the scene into account. In this paper, a robust new RGB-D SLAM method with dynamic area detection towards dynamic environments named GMSK-SLAM is proposed. Most of the existing related papers use the method of directly eliminating the whole dynamic targets. Although rejecting dynamic objects can increase the accuracy of robot positioning to a certain extent, this type of algorithm will result in the reduction of the number of available feature points in the image. The lack of sufficient feature points will seriously affect the subsequent precision of positioning and mapping for feature-based SLAM. The proposed GMSK-SLAM method innovatively combines Grid-based Motion Statistics (GMS) feature points matching method with K-means cluster algorithm to distinguish dynamic areas from the images and retain static information from dynamic environments, which can effectively increase the number of reliable feature points and keep more environment features. This method can achieve a highly improvements on localization accuracy in dynamic environments. Finally, sufficient experiments were conducted on the public TUM RGB-D dataset. Compared with ORB-SLAM2 and the RGB-D SLAM, our system, respectively, got 97.3% and 90.2% improvements in dynamic environments localization evaluated by root-mean-square error. The empirical results show that the proposed algorithm can eliminate the influence of the dynamic objects effectively and achieve a comparable or better performance than state-of-the-art methods.

     

完全在线的双目直接法视觉SLAM算法

视觉SLAM在机器人的室外作业如野外探索、定位侦察中扮演了重要角色.为了使得机器人可以更好地进行室外作业,提出一种不受词袋模型的固定词汇限制的完全在线实时双目直接法视觉SLAM算法.作为直接法视觉SLAM,所提到的系统可以利用任何具有足够强度梯度的图像像素,使其在缺少特征点的区域仍具有很强的鲁棒性.在系统算法中引入双目静态残差约束并去除遮挡的滑窗优化来增强系统的跟踪精度,增加闭环检测和位姿图优化模块,并建立在线词袋模型,使得系统在大规模且陌生的环境中依然可以进行工作.将此算法在公开的EuRoC数据集和KITTI数据集上进行性能评估,结果表明,所提出的系统的定位精度优于最先进的直接法视觉SLAM系统,且室内场景和室外场景均具有鲁棒性.     

Simultaneous localization and mapping: part I

This paper describes the simultaneous localization and mapping (SLAM) problem and the essential methods for solving the SLAM problem and summarizes key implementations and demonstrations of the method. While there are still many practical issues to overcome, especially in more complex outdoor environments, the general SLAM method is now a well understood and established part of robotics. Another part of the tutorial summarized more recent works in addressing some of the remaining issues in SLAM, including computation, feature representation, and data association.     

Real-time hierarchical stereo Visual SLAM in large-scale environments

In this paper we present a new real-time hierarchical (topological/metric) Visual SLAM system focusing on the localization of a vehicle in large-scale outdoor urban environments. It is exclusively based on the visual information provided by a cheap wide-angle stereo camera. Our approach divides the whole map into local sub-maps identified by the so-called fingerprints (vehicle poses). At the sub-map level (low level SLAM), 3D sequential mapping of natural landmarks and the robot location/orientation are obtained using a top-down Bayesian method to model the dynamic behavior. A higher topological level (high level SLAM) based on fingerprints has been added to reduce the global accumulated drift, keeping real-time constraints. Using this hierarchical strategy, we keep the local consistency of the metric sub-maps, by mean of the EKF, and global consistency by using the topological map and the MultiLevel Relaxation (MLR) algorithm. Some experimental results for different large-scale outdoor environments are presented, showing an almost constant processing time.     

Collision-free Autonomous Navigation of A Small UAV Using Low-cost Sensors in GPS-denied Environments

This paper proposes a novel complete navigation system for autonomous flight of small unmanned aerial vehicles (UAVs) in GPS-denied environments. The hardware platform used to test the proposed algorithm is a small, custom-built UAV platform equipped with an onboard computer, RGB-D camera, 2D light detection and ranging (LiDAR), and altimeter. The error-state Kalman filter (ESKF) based on the dynamic model for low-cost IMU-driven systems is proposed, and visual odometry from the RGB-D camera and height measurement from the altimeter are fed into the measurement update process of the ESKF. The pose output of the ESKF is then integrated into the open-source simultaneous location and mapping (SLAM) algorithm for pose-graph optimization and loop closing. In addition, the computationally efficient collision-free path planning algorithm is proposed and verified through simulations. The software modules run onboard in real time with limited onboard computational capability. The indoor flight experiment demonstrates that the proposed system for small UAVs with low-cost devices can navigate without collision in fully autonomous missions while establishing accurate surrounding maps.

     

A robust loop-closure method for visual SLAM in unstructured seafloor environments

This paper addresses the problem of visual simultaneous localization and mapping (SLAM) in an unstructured seabed environment that can be applied to an unmanned underwater vehicle equipped with a single monocular camera as the main measurement sensor. Monocular vision is regarded as an efficient sensing option in the context of SLAM, however it poses a variety of challenges when the relative motion is determined by matching a pair of images in the case of in-water visual SLAM. Among the various challenges, this research focuses on the problem of loop-closure which is one of the most important issues in SLAM. This study proposes a robust loop-closure algorithm in order to improve operational performance in terms of both navigation and mapping by efficiently reconstructing image matching constraints. To demonstrate and evaluate the effectiveness of the proposed loop-closure method, experimental datasets obtained in underwater environments are used, and the validity of the algorithm is confirmed by a series of comparative results.     

SLAM for autonomous planetary rovers with global localization

This paper describes a novel approach to simultaneous localization and mapping (SLAM) techniques applied to the autonomous planetary rover exploration scenario to reduce both the relative and absolute localization errors, using two well‐proven techniques: particle filters and scan matching. Continuous relative localization is improved by matching high‐resolution sensor scans to the online created local map. Additionally, to avoid issues with drifting localization, absolute localization is globally corrected at discrete times, according to predefined event criteria, by matching the current local map to the orbiter's global map. The resolutions of local and global maps can be appropriately chosen for computation and accuracy purposes. Further, the online generated local map, of the form of a structured elevation grid map, can also be used to evaluate the traversability of the surrounding environment and allow for continuous navigation. The objective of this study is to support long‐range low‐supervision planetary exploration. The implemented SLAM technique has been validated with a data set acquired during a field test campaign performed at the Teide Volcano on the island of Tenerife, representative of a Mars/Moon exploration scenario.     

Image-based flight control of unmanned aerial vehicles (UAVs) for material handling in custom manufacturing

This paper introduces an approach for and the challenges in employing unmanned aerial vehicles (UAVs) for material handling in the emerging industrial custom manufacturing environments. Compared with conventional industrial robotic systems, UAVs offer enhanced flexibility for the design and on-the-fly variation of the pathways and workflow to optimally perform multiple tasks on demand, besides offering favorable cost and dimensional footprint factors. A fundamental challenge to the deployment of UAVs in manufacturing and other indoor industrial settings lies in ensuring the accuracy of a drone’s localization and flight path. Earlier approaches based on using multiple sensors (e.g., GPS, IMU) to improve the localization accuracy of UAVs are considered ineffective in indoor environments. In fact, few investigations have tackled the issues arising due to the limited space and complicated components and moving entities, human presence in shop-floor environments. Towards addressing this challenge, a pose estimation method that employs just a single camera onboard with a UAV, together with multiple ArUco markers positioned strategically over the shop-floor is implemented to track the real-time location of a UAV. A Kalman filter is applied to mitigate noise effects for pose estimation. To assess the performance of this method, several experiments were carried out in Texas A&M University’s manufacturing labs. The result suggests that Kalman filter can reduce the variance of pose estimation by 88.48 % compared to a conventional camera and marker-based motion tracking method (∼ 27 cm), and can localize (via averaging) the position to within 8 cm of the actual target location.     

Multiple‐Robot Simultaneous Localization and Mapping: A Review

Simultaneous localization and mapping (SLAM) in unknown GPS‐denied environments is a major challenge for researchers in the field of mobile robotics. Many solutions for single‐robot SLAM exist; however, moving to a platform of multiple robots adds many challenges to the existing problems. This paper reviews state‐of‐the‐art multiple‐robot systems, with a major focus on multiple‐robot SLAM. Various issues and problems in multiple‐robot SLAM are introduced, current solutions for these problems are reviewed, and their advantages and disadvantages are discussed.     

Local map fusion for real‐time indoor simultaneous localization and mapping

Among the solutions to the simultaneous localization and mapping (SLAM) problem with probabilistic techniques, the extended Kalman filter (EKF) is a very common approach. There are several approaches to deal with its computational cost, usually based on an adequate selection of features to be updated in real time, while the whole map update is delayed or processed in a background task, allowing one to map larger environments or to carry out multirobot experiments. Although these solutions are theoretically sound, there is a great lack of real experiments in large indoor environments due to several previously unknown problems derived from the geometric model of the map features and the inconsistency of the SLAM‐EKF algorithm. For the first time, these problems are described and solved, and the implementation of the algorithms and solutions presented in this paper achieve excellent results in experiments in different real large indoor environments. © 2006 Wiley Periodicals, Inc.     

基于粒子滤波的单目视觉SLAM算法

针对携带有单目摄像机和码盘的微小机器人的定位与建图问题,提出了基于粒子滤波的SLAM(同时定位与建图)算法.从摄像机中提取图像特征点,并在图像序列中加以匹配,根据相应时刻的摄像机位姿计算得到对应的环境标志点坐标;机器人的大致位姿估计由码盘运动模型获得.在机器人移动过程中,环境标志点的观测信息和码盘信息通过粒子滤波相融合,从而提高了机器人定位的精度,同时也得到了更为准确的环境标志点坐标.仿真实验结果表明本算法有效、可靠.     

Channel invariant online visibility enhancement for visual SLAM in a turbid environment

This paper presents a real‐time and channel‐invariant visibility enhancement algorithm using a hybrid image enhancement approach. The proposed method is initially motivated by an underwater visual simultaneous localization and mapping (SLAM) failure in a turbid medium. The environments studied contain various particles and are dominated by a different image degradation model. Targeting image enhancement for degraded images but not being limited to it, the proposed method provides a highly effective solution for both color and gray images with substantial improvement in the process time compared to conventional methods. The proposed method introduces a hybrid scheme of two image enhancement modules: a model‐based (extensive) enhancement and a model‐free (immediate) enhancement. The proposed method is validated by using simulated synthetic color images and real‐world color and grayscale underwater images. Real‐world validation is performed in various environments such as hazy indoor, smoky indoor, and underwater. Using the ground truth trajectory or clear images acquired from the same area but without turbidity, we evaluate the proposed visibility enhancement and camera registration improvement for a feature based (ORB‐SLAM2), a direct (LSD‐SLAM), and a visual underwater SLAM application.