Monocular Vision for Long‐term Micro Aerial Vehicle State Estimation: A Compendium

The recent technological advances in Micro Aerial Vehicles (MAVs) have triggered great interest in the robotics community, as their deployability in missions of surveillance and reconnaissance has now become a realistic prospect. The state of the art, however, still lacks solutions that can work for a long duration in large, unknown, and GPS‐denied environments. Here, we present our visual pipeline and MAV state‐estimation framework, which uses feeds from a monocular camera and an Inertial Measurement Unit (IMU) to achieve real‐time and onboard autonomous flight in general and realistic scenarios. The challenge lies in dealing with the power and weight restrictions onboard a MAV while providing the robustness necessary in real and long‐term missions. This article provides a concise summary of our work on achieving the first onboard vision‐based power‐on‐and‐go system for autonomous MAV flights. We discuss our insights on the lessons learned throughout the different stages of this research, from the conception of the idea to the thorough theoretical analysis of the proposed framework and, finally, the real‐world implementation and deployment. Looking into the onboard estimation of monocular visual odometry, the sensor fusion strategy, the state estimation and self‐calibration of the system, and finally some implementation issues, the reader is guided through the different modules comprising our framework. The validity and power of this framework are illustrated via a comprehensive set of experiments in a large outdoor mission, demonstrating successful operation over flights of more than 360 m trajectory and 70 m altitude change. ¹      

Autonomous Navigation for Micro Aerial Vehicles in Complex GNSS-denied Environments

Micro aerial vehicles, such as multirotors, are particular well suited for the autonomous monitoring, inspection, and surveillance of buildings, e.g., for maintenance in industrial plants. Key prerequisites for the fully autonomous operation of micro aerial vehicles in restricted environments are 3D mapping, real-time pose tracking, obstacle detection, and planning of collision-free trajectories. In this article, we propose a complete navigation system with a multimodal sensor setup for omnidirectional environment perception. Measurements of a 3D laser scanner are aggregated in egocentric local multiresolution grid maps. Local maps are registered and merged to allocentric maps in which the MAV localizes. For autonomous navigation, we generate trajectories in a multi-layered approach: from mission planning over global and local trajectory planning to reactive obstacle avoidance. We evaluate our approach in a GNSS-denied indoor environment where multiple collision hazards require reliable omnidirectional perception and quick navigation reactions.     

依靠自身传感器的室内无人机自主导航引导技术综述

具有广泛应用前景的微型无人机已成为各国学者的研究热点,而不依赖卫星导航系统的室内微型无人机自主导航引导技术是研究重点之一。结合近年国内外室内无人机自主导航引导技术发展情况,讨论了依靠自身传感器实现自主导航引导的关键技术问题,详细分析了无人机位姿的解算、无人机动态避障和同步定位与地图构建的关键技术的实现情况及其难点。最后,对室内无人机自主导航引导技术进行了展望。     

Team NimbRo at MBZIRC 2017: Fast landing on a moving target and treasure hunting with a team of micro aerial vehicles

The Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2017 has defined ambitious new benchmarks to advance the state‐of‐the‐art in autonomous operation of ground‐based and flying robots. This study covers our approaches to solve the two challenges that involved micro aerial vehicles (MAV). Challenge 1 required reliable target perception, fast trajectory planning, and stable control of an MAV to land on a moving vehicle. Challenge 3 demanded a team of MAVs to perform a search and transportation task, coined “Treasure Hunt,” which required mission planning and multirobot coordination as well as adaptive control to account for the additional object weight. We describe our base MAV setup and the challenge‐specific extensions, cover the camera‐based perception, explain control and trajectory‐planning in detail, and elaborate on mission planning and team coordination. We evaluated our systems in simulation as well as with real‐robot experiments during the competition in Abu Dhabi. With our system, we—as part of the larger team NimbRo—won the MBZIRC Grand Challenge and achieved a third place in both subchallenges involving flying robots.     

PIXHAWK: A micro aerial vehicle design for autonomous flight using onboard computer vision

We describe a novel quadrotor Micro Air Vehicle (MAV) system that is designed to use computer vision algorithms within the flight control loop. The main contribution is a MAV system that is able to run both the vision-based flight control and stereo-vision-based obstacle detection parallelly on an embedded computer onboard the MAV. The system design features the integration of a powerful onboard computer and the synchronization of IMU-Vision measurements by hardware timestamping which allows tight integration of IMU measurements into the computer vision pipeline. We evaluate the accuracy of marker-based visual pose estimation for flight control and demonstrate marker-based autonomous flight including obstacle detection using stereo vision. We also show the benefits of our IMU-Vision synchronization for egomotion estimation in additional experiments where we use the synchronized measurements for pose estimation using the 2pt+gravity formulation of the PnP problem.     

RANGE–Robust autonomous navigation in GPS‐denied environments

This paper addresses the problem of autonomous navigation of a micro air vehicle (MAV) in GPS‐denied environments. We present experimental validation and analysis for our system that enables a quadrotor helicopter, equipped with a laser range finder sensor, to autonomously explore and map unstructured and unknown environments. The key challenge for enabling GPS‐denied flight of a MAV is that the system must be able to estimate its position and velocity by sensing unknown environmental structure with sufficient accuracy and low enough latency to stably control the vehicle. Our solution overcomes this challenge in the face of MAV payload limitations imposed on sensing, computational, and communication resources. We first analyze the requirements to achieve fully autonomous quadrotor helicopter flight in GPS‐denied areas, highlighting the differences between ground and air robots that make it difficult to use algorithms developed for ground robots. We report on experiments that validate our solutions to key challenges, namely a multilevel sensing and control hierarchy that incorporates a high‐speed laser scan‐matching algorithm, data fusion filter, high‐level simultaneous localization and mapping, and a goal‐directed exploration module. These experiments illustrate the quadrotor helicopter's ability to accurately and autonomously navigate in a number of large‐scale unknown environments, both indoors and in the urban canyon. The system was further validated in the field by our winning entry in the 2009 International Aerial Robotics Competition, which required the quadrotor to autonomously enter a hazardous unknown environment through a window, explore the indoor structure without GPS, and search for a visual target. © 2011 Wiley Periodicals, Inc.     

Autonomous Navigation of UAV in Foliage Environment

This paper presents a navigation system that enables small-scale unmanned aerial vehicles to navigate autonomously using a 2D laser range finder in foliage environment without GPS. The navigation framework consists of real-time dual layer control, navigation state estimation and online path planning. In particular, the inner loop of a quadrotor is stabilized using a commercial autopilot while the outer loop control is implemented using robust perfect tracking. The navigation state estimation consists of real-time onboard motion estimation and trajectory smoothing using the GraphSLAM technique. The onboard real-time motion estimation is achieved by a Kalman filter, fusing the planar velocity measurement from matching the consecutive scans of a laser range finder and the acceleration measurement of an inertial measurement unit. The trajectory histories from the real-time autonomous navigation together with the observed features are fed into a sliding-window based pose-graph optimization framework. The online path planning module finds an obstacle-free trajectory based the local measurement of the laser range finder. The performance of the proposed navigation system is demonstrated successfully on the autonomous navigation of a small-scale UAV in foliage environment.     

The ETH‐MAV Team in the MBZ International Robotics Challenge

This study describes the hardware and software systems of the Micro Aerial Vehicle (MAV) platforms used by the ETH Zurich team in the 2017 Mohamed Bin Zayed International Robotics Challenge (MBZIRC). The aim was to develop robust outdoor platforms with the autonomous capabilities required for the competition, by applying and integrating knowledge from various fields, including computer vision, sensor fusion, optimal control, and probabilistic robotics. This paper presents the major components and structures of the system architectures and reports on experimental findings for the MAV‐based challenges in the competition. Main highlights include securing the second place both in the individual search, pick, and place the task of Challenge 3 and the Grand Challenge, with autonomous landing executed in less than 1 min and a visual servoing success rate of over for object pickups.     

Control architecture for mobile robot operation and navigation

This paper presents the navigation and operation system (NOS) for a multipurpose industrial autonomous mobile robot for both indoor and outdoor environments. This architecture supports task specification in terms of an event-driven state-based machine that provides high quality mission performance in uncertain environments. All processes in the NOS have been integrated in a distributed architecture designed to consider the real-time constraints of each control level of the system. Particular task models obtained from the system requirements specifications are integrated at the highest level of the architecture so that the rest of the levels remain unchanged for a wide range of industrial applications, such as transportation and operation with onboard devices.     

An open‐source system for vision‐based micro‐aerial vehicle mapping,planning, and flight in cluttered environments

We present an open‐source system for Micro‐Aerial Vehicle (MAV) autonomous navigation from vision‐based sensing. Our system focuses on dense mapping, safe local planning, and global trajectory generation, especially when using narrow field‐of‐view sensors in very cluttered environments. In addition, details about other necessary parts of the system and special considerations for applications in real‐world scenarios are presented. We focus our experiments on evaluating global planning, path smoothing, and local planning methods on real maps made on MAVs in realistic search‐and‐rescue and industrial inspection scenarios. We also perform thousands of simulations in cluttered synthetic environments, and finally validate the complete system in real‐world experiments.     

A Biologically-Inspired Micro Aerial Vehicle

This paper introduces a novel framework for the design, modeling and control of a Micro Aerial Vehicle (MAV). The vehicle’s conceptual design is based on biologically-inspired principles and emulates a dragonfly (Odonata–Anisoptera). We have taken inspiration from the flight mechanism features of the dragonfly and have developed indigenous designs in creating a novel version of a Flapping Wing MAV (FWMAV). The MAV design incorporates a complex mechanical construction and a sophisticated multi-layered, hybrid, linear/non-linear controller to achieve extended flight times and improved agility compared to other rotary wing and FWMAV Vertical Take Off and Landing (VTOL) designs. The first MAV prototype will have a ballpark weight including sensor payload of around 30 g. The targeted lifting capability is about twice the weight. The MAV features state of the art sensing and instrumentation payload, which includes integrated high-power on-board processors, 6DoF inertial sensors, 3DoF compasses, GPS, embedded camera and long-range telemetry capability. A 3-layer control mechanism has been developed to harness the dynamics and attain complete navigational control of the MAV. The inner-layer is composed of a ‘quad hybrid-energy controller’ and two higher layers are at present, implementing a linear controller; the latter will be replaced eventually with a dynamic adaptive non-linear controller. The advantages of the proposed design compared to other similar ones include higher energy efficiency and extended flight endurance. The design features elastic storage and re-use of propulsion energy favoring energy conservation during flight. The design/modeling of the MAV and its kinematics & dynamics have been tested under simulation to achieve desired performance. The potential applications for such a high endurance vehicle are numerous, including air-deployable mass surveillance and reconnaissance in cluster and swarm formations. The efficacy of the design is demonstrated through a simulation environment. The dynamics are verified through simulations and a general linear controller coupled with an energy based non-linear controller is shown to operate the vehicle in a stable regime. In accordance with specified objectives a prototype is being developed for flight-testing and demonstration purposes.     

激光雷达集成电路存储单元上电复位状态机设计

为了提高激光雷达集成电路存储单元上电复位的稳定性,提出基于集成DSP的激光雷达集成电路存储单元上电复位状态机设计方法。构建激光雷达集成电路存储单元复位状态机的总体结构模型,采用内核电源电路进行单极点高通滤波控制,通过由DSP集成信息模块等组成的上电复位机控制的方法,进行激光雷达集成电路存储单元的程序加载和基线恢复控制,通过基线恢复器实现激光雷达集成电路存储单元的掉电复位和连通性测试,实现激光雷达集成电路的逻辑时序控制和上电状态机设计,实现激光雷达集成电路存储单元的硬件优化设计。仿真结果表明,设计的激光雷达集成电路存储单元上电复位状态机稳定性较好,集成控制性能较强,提高了激光雷达集成信号采集能力。     

Development of a Rotorcraft Micro air Vehicle for Indoor Flight Research

Autonomous flight of micro air vehicles (MAVs) in hostile indoor environments poses significant challenges in terms of control and navigation. In order to support navigation and control research for indoor micro air vehicles, a four-wing tail-sitter type rotorcraft MAV weighing less than 350g has been designed in this paper. In an effort to achieve autonomous indoor flight, an embedded integrated avionic system has been developed. The modeling process has been conducted to obtain accurate six degrees of freedom dynamical model for the designed rotorcraft MAV. In addition, aerodynamic coefficients are evaluated from the results of Computational Fluid Dynamics A PI-ADRC double loop controller with inner-loop outer-loop control scheme has been proposed which takes into account the system’s nonlinearities and uncertainties. The proposed flight controller was implemented on the designed rotorcraft MAV that has undergone various simulation and indoor flight tests. Experimental results that demonstrate robustness of the proposed controller with respect to external disturbances and the capabilities of the designed rotorcraft MAV are presented.     

Inventing a Biologically Inspired, Energy Efficient Micro Aerial Vehicle

In recent years, research efforts have focused on the design, development and deployment of unmanned systems for a variety of applications ranging from intelligence and surveillance to border patrol, rescue operations, etc. Micro Aerial Vehicles are viewed as potential targets that can provide agility and accurate small area coverage while being cost-effective and can be easily launched by a single operator. The small size of MAVs allows such flight operations within confined space but the control effectors must provide sufficient maneuverability, while maintaining stability, with only limited sensing capability onboard the platform. To meet these challenges, researchers have long been attracted by the amazing attributes of biological systems, such as those exhibited by birds and insects. Birds can fly in dense flocks, executing rapid maneuvers with g-loads far in excess of modern fighter aircrafts, and yet never collide with each other, despite the absence of air traffic controllers. This paper introduces a novel framework for the design and control of a Micro Air Vehicle. The vehicle’s conceptual design is based on biologically-inspired principles and emulates a dragonfly (Odonata–Anisoptera). A sophisticated multi-layered Hybrid & Linear/Non-Linear controller to achieve extended flight times and improved agility compared to other Rotary and Flapping Wing MAV designs. The paper addresses the design and control features of the proposed QV design and gives an overview on the developmental efforts towards the prototyping of the flyer. The potential applications for such a high endurance vehicle are numerous, including air-deployable mass surveillance in cluster and swarm formations. The disposable nature of the vehicle would help in battle-field deployment as well, where such a MAV would be made available to soldiers for proximity sensing and threat level assessment. Other applications would include search and rescue operations and civilian law-enforcement.     

面向室内场景的主被动融合视觉定位系统

目的 视觉定位旨在利用易于获取的RGB图像对运动物体进行目标定位及姿态估计。室内场景中普遍存在的物体遮挡、弱纹理区域等干扰极易造成目标关键点的错误估计,严重影响了视觉定位的精度。针对这一问题,本文提出一种主被动融合的室内定位系统,结合固定视角和移动视角的方案优势,实现室内场景中运动目标的精准定位。方法 提出一种基于平面先验的物体位姿估计方法,在关键点检测的单目定位框架基础上,使用平面约束进行3自由度姿态优化,提升固定视角下室内平面中运动目标的定位稳定性。基于无损卡尔曼滤波算法设计了一套数据融合定位系统,将从固定视角得到的被动式定位结果与从移动视角得到的主动式定位结果进行融合,提升了运动目标的位姿估计结果的可靠性。结果 本文提出的主被动融合室内视觉定位系统在iGibson仿真数据集上的平均定位精度为2～3 cm,定位误差在10 cm内的准确率为99%;在真实场景中平均定位精度为3～4 cm,定位误差在10 cm内的准确率在90%以上,实现了cm级的定位精度。结论 提出的室内视觉定位系统融合了被动式和主动式定位方法的优势,能够以较低设备成本实现室内场景中高精度的目标定位结果,并在遮挡、目标...     

微型飞行器研究进展与关键技术

微型飞行器(MAV)突破了传统常规飞行器的设计和制作概念,是随着微米纳米科技和微电子机械系统(MEMS)的蓬勃兴起而发展起来的一个新的研究领域,国际上是一个研究热点。介绍了国内外各种MAV当前的研究发展状况,分析了MAV研制中的一些关键技术。     

Design of a 3D snapshot based visual flight control system using a single camera in hover

The problem of developing a reliable system for sensing and controlling the hover of a Micro Air Vehicle (MAV) using visual snapshots is considered. The current problem is part of a larger project, which is developing an autonomous MAV, controlled by vision only information. A new algorithm is proposed that uses a stored image of the ground, a snapshot taken of the ground directly under the MAV, as a visual anchor point. The absolute translation of the aircraft and its velocity are then calculated by comparing the subsequent frames with the stored image and fed into the position controller. In order to increase the performance, several issues, such as effects of scale uncertainty on the closed loop stability of the platform are investigated. For controller design and testing purposes, we analytically derive a complete model of a small size helicopter with no stabilizing bar (flybar). The simulation results for 2D and 3D snapshots confirm the effectiveness of the proposed algorithm.     

Multisensor-Based Human Detection and Tracking for Mobile Service Robots

One of fundamental issues for service robots is human-robot interaction. In order to perform such a task and provide the desired services, these robots need to detect and track people in the surroundings. In this paper, we propose a solution for human tracking with a mobile robot that implements multisensor data fusion techniques. The system utilizes a new algorithm for laser-based leg detection using the onboard laser range finder (LRF). The approach is based on the recognition of typical leg patterns extracted from laser scans, which are shown to also be very discriminative in cluttered environments. These patterns can be used to localize both static and walking persons, even when the robot moves. Furthermore, faces are detected using the robot's camera, and the information is fused to the legs' position using a sequential implementation of unscented Kalman filter. The proposed solution is feasible for service robots with a similar device configuration and has been successfully implemented on two different mobile platforms. Several experiments illustrate the effectiveness of our approach, showing that robust human tracking can be performed within complex indoor environments.     

微小型飞行器航向系统设计

从实用的角度出发,给出了微小型飞行器的航向系统(包括航向测量和航向控制)体系结构．设计了地磁航向、捷联航向、GPS航向等3种常用的航向测量手段在微小型系统中的实现方案,分析了其各自的优、缺点,提出了一种实用的、基于数据融合的航向获取方法．在航向测量的基础上,又设计了针对微小型飞行器的航向控制方案,并应用于某小型固定翼飞行器的飞行控制系统中．进行了自主飞行试验,取得了预期的试验效果．