Vision-based target tracking with a small UAV: Optimization-based control strategies

This paper considers the problem of a small, fixed-wing UAV equipped with a gimbaled camera autonomously tracking an unpredictable moving ground vehicle. Thus, the UAV must maintain close proximity to the ground target and simultaneously keep the target in its camera׳s visibility region. To achieve this objective robustly, two novel optimization-based control strategies are developed. The first assumes an evasive target motion while the second assumes a stochastic target motion. The resulting optimal control policies have been successfully flight tested, thereby demonstrating the efficacy of both approaches in a real-world implementation and highlighting the advantages of one approach over the other.     

一种小型无人机地空跟踪监视系统设计与实现

针对无人机地面监视系统的功能要求和存在的问题,设计了一种小型无人机地空跟踪监视系统并给出了系统的结构与工作原理.系统以嵌入式主板为核心,采用基于GPS的自动引导方式为主、以航迹预测和图像处理为辅助的多种跟踪监视模式实现了无人机飞行过程的全程监视.对跟踪方法、GPS静态漂移、云台镜头控制等关键问题进行了分析,给出了设计方...     

Tracking a Ground Moving Target with a Quadrotor Using Switching Control

An unmanned aerial vehicle (UAV) stabilization strategy based on computer vision and switching controllers is proposed. The main goal of this system is to perform tracking of a moving target on ground. The architecture implemented consists of a quadrotor equipped with an embedded camera which provides real-time video to a computer vision algorithm where images are processed. A vision-based estimator is proposed, which makes use of 2-dimensional images to compute the relative 3-dimensional position and translational velocity of the UAV with respect to the target. The proposed estimator provides the required states measurements to a micro-controller for stabilizing the vehicle during flight. The control strategy consists of switching controllers, which allows making decisions when the target is lost temporarily or when it is out of the camera’s field of view. Real time experiments are presented to demonstrate the performance of the target-tracking system proposed.     

无人机多机动目标自主探测与跟踪

无人机机载相机图像中机动目标尺寸较小而且会发生显著变化,加上大量的背景噪声干扰,给目标探测和跟踪带来很大困难.针对这些问题,本文提出了一种在无人机机载相机图像序列中自主探测与跟踪多个机动目标的方法.首先,提取目标的图像数字特征并采用级联分类算法进行特征分类,得到目标的强分类器,对目标进行自主探测搜索.然后,基于全局最优关联算法对探测回波进行关联滤波,实现对多个机动目标的跟踪与识别,其中最优关联代价矩阵融合了距离和方向信息,提高了关联和跟踪的鲁棒性.将无人机航拍图像序列中的地面坦克作为目标进行实验,结果表明本文算法可以实现对多个机动目标的自主探测和跟踪,并具有较好的跟踪鲁棒性.     

Augmenting UAV autonomy

The addition of a camera to an unmanned aerial vehicle (UAV) allows the vehicle to perform a variety of tasks autonomously. This paper presents UAV vision systems that were developed and tested at the Georgia Institute of Technology to perform three of these tasks on the GTMax unmanned research helicopter. The first system demonstrates a vision-based navigation system. The second system demonstrates an automated search routine for stationary ground targets. The third system demonstrates a ground target tracking architecture for mobile targets.     

Unmanned aerial vehicles UAVs attitude,height, motion estimation and control using visual systems

This paper presents an implementation of an aircraft pose and motion estimator using visual systems as the principal sensor for controlling an Unmanned Aerial Vehicle (UAV) or as a redundant system for an Inertial Measure Unit (IMU) and gyros sensors. First, we explore the applications of the unified theory for central catadioptric cameras for attitude and heading estimation, explaining how the skyline is projected on the catadioptric image and how it is segmented and used to calculate the UAV’s attitude. Then we use appearance images to obtain a visual compass, and we calculate the relative rotation and heading of the aerial vehicle. Additionally, we show the use of a stereo system to calculate the aircraft height and to measure the UAV’s motion. Finally, we present a visual tracking system based on Fuzzy controllers working in both a UAV and a camera pan and tilt platform. Every part is tested using the UAV COLIBRI platform to validate the different approaches, which include comparison of the estimated data with the inertial values measured onboard the helicopter platform and the validation of the tracking schemes on real flights.     

Design of small hand‐launched solar‐powered UAVs: From concept study to a multi‐day world endurance record flight

We present the development process behind AtlantikSolar, a small 6.9 kg hand‐launchable low‐altitude solar‐powered unmanned aerial vehicle (UAV) that recently completed an 81‐hour continuous flight and thereby established a new flight endurance world record for all aircraft below 50 kg mass. The goal of our work is to increase the usability of such solar‐powered robotic aircraft by maximizing their perpetual flight robustness to meteorological deteriorations such as clouds or winds. We present energetic system models and a design methodology, implement them in our publicly available conceptual design framework for perpetual flight‐capable solar‐powered UAVs, and finally apply the framework to the AtlantikSolar UAV. We present the detailed AtlantikSolar characteristics as a practical design example. Airframe, avionics, hardware, state estimation, and control method development for autonomous flight operations are described. Flight data are used to validate the conceptual design framework. Flight results from the continuous 81‐hour and 2,338 km covered ground distance flight show that AtlantikSolar achieves 39% minimum state‐of‐charge, 6.8 h excess time and 6.2 h charge margin. These performance metrics are a significant improvement over previous solar‐powered UAVs. A performance outlook shows that AtlantikSolar allows perpetual flight in a 6‐month window around June 21 at mid‐European latitudes, and that multi‐day flights with small optical‐ or infrared‐camera payloads are possible for the first time. The demonstrated performance represents the current state‐of‐the‐art in solar‐powered low‐altitude perpetual flight performance. We conclude with lessons learned from the three‐year AtlantikSolar UAV development process and with a sensitivity analysis that identifies the most promising technological areas for future solar‐powered UAV performance improvements.     

An adaptive vision-based autopilot for mini flying machines guidance,navigation and control

The design of reliable navigation and control systems for Unmanned Aerial Vehicles (UAVs) based only on visual cues and inertial data has many unsolved challenging problems, ranging from hardware and software development to pure control-theoretical issues. This paper addresses these issues by developing and implementing an adaptive vision-based autopilot for navigation and control of small and mini rotorcraft UAVs. The proposed autopilot includes a Visual Odometer (VO) for navigation in GPS-denied environments and a nonlinear control system for flight control and target tracking. The VO estimates the rotorcraft ego-motion by identifying and tracking visual features in the environment, using a single camera mounted on-board the vehicle. The VO has been augmented by an adaptive mechanism that fuses optic flow and inertial measurements to determine the range and to recover the 3D position and velocity of the vehicle. The adaptive VO pose estimates are then exploited by a nonlinear hierarchical controller for achieving various navigational tasks such as take-off, landing, hovering, trajectory tracking, target tracking, etc. Furthermore, the asymptotic stability of the entire closed-loop system has been established using systems in cascade and adaptive control theories. Experimental flight test data over various ranges of the flight envelope illustrate that the proposed vision-based autopilot performs well and allows a mini rotorcraft UAV to achieve autonomously advanced flight behaviours by using vision.     

Autonomous Mobile Robot Localization and Navigation Using a Hierarchical Map Representation Primarily Guided by Vision

While impressive progress has recently been made with autonomous vehicles, both indoors and on streets, autonomous localization and navigation in less constrained and more dynamic environments, such as outdoor pedestrian and bicycle‐friendly sites, remains a challenging problem. We describe a new approach that utilizes several visual perception modules—place recognition, landmark recognition, and road lane detection—supplemented by proximity cues from a planar laser range finder for obstacle avoidance. At the core of our system is a new hybrid topological/grid‐occupancy map that integrates the outputs from all perceptual modules, despite different latencies and time scales. Our approach allows for real‐time performance through a combination of fast but shallow processing modules that update the map's state while slower but more discriminating modules are still computing. We validated our system using a ground vehicle that autonomously traversed three outdoor routes several times, each 400 m or longer, on a university campus. The routes featured different road types, environmental hazards, moving pedestrians, and service vehicles. In total, the robot logged over 10 km of successful recorded experiments, driving within a median of 1.37 m laterally of the center of the road, and localizing within 0.97 m (median) longitudinally of its true location along the route.     

Autonomous landing on a moving vehicle with an unmanned aerial vehicle

This paper addresses the perception, control, and trajectory planning for an aerial platform to identify and land on a moving car at 15 km/hr. The hexacopter unmanned aerial vehicle (UAV), equipped with onboard sensors and a computer, detects the car using a monocular camera and predicts the car future movement using a nonlinear motion model. While following the car, the UAV lands on its roof, and it attaches itself using magnetic legs. The proposed system is fully autonomous from takeoff to landing. Numerous field tests were conducted throughout the year‐long development and preparations for the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2017 competition, for which the system was designed. We propose a novel control system in which a model predictive controller is used in real time to generate a reference trajectory for the UAV, which are then tracked by the nonlinear feedback controller. This combination allows to track predictions of the car motion with minimal position error. The evaluation presents three successful autonomous landings during the MBZIRC 2017, where our system achieved the fastest landing among all competing teams.     

Standoff Tracking of a Moving Target for Quadrotor Using Lyapunov Potential Function

International Journal of Control, Automation and Systems - This paper presents a control scheme for standoff tracking of a ground moving target by a quadrotor unmanned aerial vehicle (UAV). The...     

Model based vehicle detection and tracking for autonomous urban driving

Situational awareness is crucial for autonomous driving in urban environments. This paper describes the moving vehicle detection and tracking module that we developed for our autonomous driving robot Junior. The robot won second place in the Urban Grand Challenge, an autonomous driving race organized by the U.S. Government in 2007. The module provides reliable detection and tracking of moving vehicles from a high-speed moving platform using laser range finders. Our approach models both dynamic and geometric properties of the tracked vehicles and estimates them using a single Bayes filter per vehicle. We present the notion of motion evidence, which allows us to overcome the low signal-to-noise ratio that arises during rapid detection of moving vehicles in noisy urban environments. Furthermore, we show how to build consistent and efficient 2D representations out of 3D range data and how to detect poorly visible black vehicles. Experimental validation includes the most challenging conditions presented at the Urban Grand Challenge as well as other urban settings.     

RANGE–Robust autonomous navigation in GPS‐denied environments

This paper addresses the problem of autonomous navigation of a micro air vehicle (MAV) in GPS‐denied environments. We present experimental validation and analysis for our system that enables a quadrotor helicopter, equipped with a laser range finder sensor, to autonomously explore and map unstructured and unknown environments. The key challenge for enabling GPS‐denied flight of a MAV is that the system must be able to estimate its position and velocity by sensing unknown environmental structure with sufficient accuracy and low enough latency to stably control the vehicle. Our solution overcomes this challenge in the face of MAV payload limitations imposed on sensing, computational, and communication resources. We first analyze the requirements to achieve fully autonomous quadrotor helicopter flight in GPS‐denied areas, highlighting the differences between ground and air robots that make it difficult to use algorithms developed for ground robots. We report on experiments that validate our solutions to key challenges, namely a multilevel sensing and control hierarchy that incorporates a high‐speed laser scan‐matching algorithm, data fusion filter, high‐level simultaneous localization and mapping, and a goal‐directed exploration module. These experiments illustrate the quadrotor helicopter's ability to accurately and autonomously navigate in a number of large‐scale unknown environments, both indoors and in the urban canyon. The system was further validated in the field by our winning entry in the 2009 International Aerial Robotics Competition, which required the quadrotor to autonomously enter a hazardous unknown environment through a window, explore the indoor structure without GPS, and search for a visual target. © 2011 Wiley Periodicals, Inc.     

Combining Stereo Vision and Inertial Navigation System for a Quad-Rotor UAV

This paper presents the development of a quad-rotor robotic platform equipped with a visual and inertial motion estimation system. Our objective consists of developing a UAV capable of autonomously perform take-off, positioning, navigation and landing in unknown environments. In order to provide accurate estimates of the UAV position and velocity, stereo visual odometry and inertial measurements are fused using a Kalman Filter. Real-time experiments consisting on motion detection and autonomous positioning demonstrate the performance of the robotic platform.     

Design and implementation of an autonomous flight control law for a UAV helicopter

In this paper, we present the design and implementation of an autonomous flight control law for a small-scale unmanned aerial vehicle (UAV) helicopter. The approach is decentralized in nature by incorporating a newly developed nonlinear control technique, namely the composite nonlinear feedback control, together with dynamic inversion. The overall control law consists of three hierarchical layers, namely, the kernel control, command generator and flight scheduling, and is implemented and verified in flight tests on the actual UAV helicopter. The flight test results demonstrate that the UAV helicopter is capable of carrying out complicated flight missions autonomously.     

Vision-Based Road-Following Using Proportional Navigation

This paper describes a new approach for autonomous road following for an unmanned air vehicle (UAV) using a visual sensor. A road is defined as any continuous, extended, curvilinear feature, which can include city streets, highways, and dirt roads, as well as forest-fire perimeters, shorelines, and fenced borders. To achieve autonomous road-following, this paper utilizes Proportional Navigation as the basis for the guidance law, where visual information is directly fed back into the controller. The tracking target for the Proportional Navigation algorithm is chosen as the position on the edge of the camera frame at which the road flows into the image. Therefore, each frame in the video stream only needs to be searched on the edge of the frame, thereby significantly reducing the computational requirements of the computer vision algorithms. The tracking error defined in the camera reference frame shows that the Proportional Navigation guidance law results in a steady-state error caused by bends and turns in the road, which are perceived as road motion. The guidance algorithm is therefore adjusted using Augmented Proportional Navigation Guidance to account for the perceived road accelerations and to force the steady-state error to zero. The effectiveness of the solution is demonstrated through high-fidelity simulations, and with flight tests using a small autonomous UAV.     

Visual Object Tracking and Servoing Control of a Nano-Scale Quadrotor: System,Algorithms, and Experiments

There are two main trends in the development of unmanned aerial vehicle (UAV) technologies: miniaturization and intellectualization, in which realizing object tracking capabilities for a nano-scale UAV is one of the most challenging problems. In this paper, we present a visual object tracking and servoing control system utilizing a tailor-made 38 g nano-scale quadrotor. A lightweight visual module is integrated to enable object tracking capabilities, and a micro positioning deck is mounted to provide accurate pose estimation. In order to be robust against object appearance variations, a novel object tracking algorithm, denoted by RMCTer, is proposed, which integrates a powerful short-term tracking module and an efficient long-term processing module. In particular, the long-term processing module can provide additional object information and modify the short-term tracking model in a timely manner. Furthermore, a position-based visual servoing control method is proposed for the quadrotor, where an adaptive tracking controller is designed by leveraging backstepping and adaptive techniques. Stable and accurate object tracking is achieved even under disturbances. Experimental results are presented to demonstrate the high accuracy and stability of the whole tracking system.      

面向无人机的视觉目标跟踪算法：综述与展望

近年来,无人机因其小巧灵活、智能自主等特点被广泛应用于民用和军事等领域中,特别是搜索侦察过程中首要的目标跟踪任务。无人机视觉目标跟踪场景的复杂性和运动目标的多变性,使得目标特征提取及模型建立困难,对目标跟踪性能带来巨大的挑战。本文首先介绍了无人机视觉目标跟踪的研究现状,梳理了经典和最新的目标跟踪算法,特别是基于相关滤波的跟踪算法和基于深度学习的跟踪算法,并对比了不同算法的优缺点。其次,归纳了常用的目标跟踪数据集和性能评价指标。最后,展望了无人机视觉目标跟踪算法的未来发展趋势。     

无人机空中加油自主会合控制器设计

针对自主空中加油中的飞行器会合问题,提出一个完整的自主控制方案,以使无人机在无加油机主动配合情况下具备自主会合能力.通过改进带终端碰撞角约束的比例导引律,将期望的会合点作为跟踪目标,实现终端航向的控制.为了满足会合对无人机机动性能的要求,使用Backstepping方法设计控制律,在保证无人机稳定性的基础上,实现了制导信号的精确跟踪.仿真结果验证了所提出控制方案的有效性.     

An on-board vision sensor system for small unmanned vehicle applications

This paper describes an on-board vision sensor system that is developed specifically for small unmanned vehicle applications. For small vehicles, vision sensors have many advantages, including size, weight, and power consumption, over other sensors such as radar, sonar, and laser range finder, etc. A vision sensor is also uniquely suited for tasks such as target tracking and recognition that require visual information processing. However, it is difficult to meet the computing needs of real-time vision processing on a small robot. In this paper, we present the development of a field programmable gate array-based vision sensor and use a small ground vehicle to demonstrate that this vision sensor is able to detect and track features on a user-selected target from frame to frame and steer the small autonomous vehicle towards it. The sensor system utilizes hardware implementations of the rank transform for filtering, a Harris corner detector for feature detection, and a correlation algorithm for feature matching and tracking. With additional capabilities supported in software, the operational system communicates wirelessly with a base station, receiving commands, providing visual feedback to the user and allowing user input such as specifying targets to track. Since this vision sensor system uses reconfigurable hardware, other vision algorithms such as stereo vision and motion analysis can be implemented to reconfigure the system for other real-time vision applications.