Randomized pursuit-evasion in a polygonal environment

This paper contains two main results. First, we revisit the well-known visibility-based pursuit-evasion problem, and show that in contrast to deterministic strategies, a single pursuer can locate an unpredictable evader in any simply connected polygonal environment, using a randomized strategy. The evader can be arbitrarily faster than the pursuer, and it may know the position of the pursuer at all times, but it does not have prior knowledge of the random decisions made by the pursuer. Second, using the randomized algorithm, together with the solution to a problem called the "lion and man problem" as subroutines, we present a strategy for two pursuers (one of which is at least as fast as the evader) to quickly capture an evader in a simply connected polygonal environment. We show how this strategy can be extended to obtain a strategy for a polygonal room with a door, two pursuers who have only line-of-sight communication, and a single pursuer (at the expense of increased capture time).     

Three-dimensional pursuer convoy by using guidance laws

This paper investigates the problem of modeling and controlling pursuer convoy in three-dimensional space. The guidance laws applied for convoy, the velocity pursuit, the deviated pursuit and the proportional navigation, steer the pursuer using the rate of line-of-sight (LOS) between successive pursuers. On the basis of the differential equations for the range, the pitch angle of LOS and the yaw angle of LOS between successive pursuers, the guidance laws are proposed to derive decentralized control strategy for pursuer convoy. The results concerning the pursuer convoy are rigorously proven. Simulations are conducted to demonstrate the feasibility and effectiveness of the proposed control strategy.     

A cooperative Homicidal Chauffeur game

We address a pursuit-evasion problem involving an unbounded planar environment, a single evader and multiple pursuers moving along curves of bounded curvature. The problem amounts to a multi-agent version of the classic Homicidal Chauffeur problem; we identify parameter ranges in which a single pursuer is not sufficient to guarantee evader capture. We propose a novel multi-phase cooperative strategy in which the pursuers move in specific formations and confine the evader to a bounded region. The proposed strategy is inspired by the hunting and foraging behaviors of various fish species. We characterize the required number of pursuers for which our strategy is guaranteed to lead to confinement.     

Cooperative Multiple Pursuers against a Single Evader

This paper considers a pursuit-evasion game for non-holonomic systems where a group of pursuers attempts to capture an evader in a bounded connected domain. The problem is challenging because all vehicles have the same maneuvering capability in terms of speed and turn radius constraint. The paper initially discusses a simple approach for holonomic systems that is based on the minimization of the safe-reachable area (the area containing the set of points to where an evader can travel without being caught). This idea is then extended to develop a pursuit-evasion strategy for non-holonomic systems. However, solving such a problem is computationally intractable. Therefore, we propose a computationally efficient algorithm to obtain approximate solutions. This paper also proposes an alternative approach to obtain a simple yet effective solution to the cooperative pursuit problem that is based on missile guidance laws. As there is no analytical proof of capture, we empirically evaluate the performance of the algorithms and perform a comparative study using solutions obtained from umpteen simulations. A total of four different cooperative pursuit strategies and three different evader strategies are taken into account for the comparative study. In the process, an evader strategy which is superior to that based on the optimization of safe-reachable area is also identified.     

A Mixed Integer Linear Programming approach to pursuit evasion problems with optional connectivity constraints

In this paper, we address the multi pursuer version of the pursuit evasion problem in polygonal environments. By discretizing the problem, and applying a Mixed Integer Linear Programming (MILP) framework, we are able to address problems requiring so-called recontamination and also impose additional constraints, such as connectivity between the pursuers. The proposed MILP formulation is less conservative than solutions based on graph discretizations of the environment, but still somewhat more conservative than the original underlying problem. It is well known that MILPs, as well as multi pursuer pursuit evasion problems, are NP-hard. Therefore we apply an iterative Receding Horizon Control (RHC) scheme where a number of smaller MILPs are solved over shorter planning horizons. The proposed approach is implemented in Matlab/Cplex and illustrated by a number of solved examples.     

Guaranteed strategies with memory for alternative pursuit

A pursuit game is called alternative if, at any state of the game, a pursuer can terminate it on any of two given target sets and the Boltz payoff function differs on these sets only in the terminal part. By assumption, the optimal universal feedback strategies of the pursuer are known for each fixed terminal alternative. Endeavoring to decrease its payoff via the choice of an alternative, the pursuer may compare guaranteed results (e.g., at the initial and current states) and apply one of two fixed-alternative strategies. If both players optimize the choice of their alternatives depending on the position, the state of the pursuit game can enter some neighborhood of a hypersurface with equivalent alternatives (in the sense of payoffs) and stay there for a finite period of time. In different formalizations, the solutions generated by an appropriate differential equation with discontinuous right-hand side (ideal solutions, limits of Euler polygonal lines, generalized solutions) lead to different outcomes. For certain states, they even increase the payoff in comparison with the pursuit strategy based on the preferable alternative chosen at the initial state (due to sliding mode occurrence). The author introduces a method of constructing pursuit strategies with memory and a finite number of admissible switchings for the target alternative. The proposed method allows estimating the guaranteed result of the pursuer by confining to the solution concept of the limits of Euler polygonal lines.     

Multi-agent cooperation pursuit based on an extension of AALAADIN organisational model

An approach of cooperative pursuit for multiple mobile targets based on multi-agents system is discussed. In this kind of problem the pursuit process is divided into two kinds of tasks. The first one (coalition problem) is designed to solve the problem of the pursuit team formation. To achieve this mission, we used an innovative method based on a dynamic organisation and reorganisation of the pursuers’ groups. We introduce our coalition strategy extended from the organisational agent, group, role model by assigning an access mechanism to the groups inspired by fuzzy logic principles. The second task (motion problem) is the treatment of the pursuers’ motion strategy. To manage this problem we applied the principles of the Markov decision process. Simulation results show the feasibility and validity of the given proposal.     

Coordinated pursuer control using particle filters for autonomous search-and-capture

In a real-world pursuit-evasion (PE) game, the pursuers often have a limited field-of-view of the evaders and thus are required to search for and detect the evaders before capturing them. This paper presents a unified framework and control algorithm using particle filters (PFs) for the coordination of multiple pursuers to search for and capture multiple evaders given the ability of PF to estimate highly non-Gaussian densities prevalent in search problems. The pursuer control problem is formulated as a stochastic control problem where global objectives function of both searching and capturing are common. To take the evaders’ actions into account, an action measure (AM) is defined over the evaders’ PDs is used to represent the probability that the evader may transit each state in the PD. The global objective functions for search and capture are then decomposed into local objective functions for unification through objective priority weights. Coordination between the pursuers takes place through the multi-sensor update where the observation likelihoods of all pursuers are used in the PF update stage. The control actions of each pursuer are then determined individually, based on the updated PDs given the objective weights, action measures as well as evader importance weights in the case of multiple evaders. The proposed algorithm is tested in three scenarios for its effectiveness. In addition, a parametric study on the average capture time against the initial variances of the target state uncertainty is conducted to test for robustness. Results show that the pursuers are able to capture all the evaders in each case with the capture time for the second and last scenario differing by only 2.9% implying firstly that under the proposed algorithm, the capture time is not proportional to the increase in the number of evaders and also suggested robustness and potential scalability of the proposed algorithm.     

On Discrete-Time Pursuit-Evasion Games With Sensing Limitations

In this paper, we address discrete-time pursuit-evasion games in the plane where every player has identical sensing and motion ranges restricted to closed disks of given sensing and stepping radii. A single evader is initially located inside a bounded subset of the environment and does not move until detected. We propose a sweep-pursuit-capture pursuer strategy to capture the evader and apply it to two variants of the game. The first involves a single pursuer and an evader in a bounded convex environment, and the second involves multiple pursuers and an evader in a boundaryless environment. In the first game, we give a sufficient condition on the ratio of sensing to stepping radius of the players that guarantees capture. In the second, we determine the minimum probability of capture, which is a function of a novel pursuer formation and independent of the initial evader location. The sweep and pursuit phases reduce both games to previously studied problems with unlimited range sensing, and capture is achieved using available strategies. We obtain novel upper bounds on the capture time and present simulation studies that address the performance of the strategies under sensing errors, different ratios of sensing to stepping radius, greater evader speed, and a different number of pursuers.      

Pursuing a maneuvering target which uses a random process for itscontrol

Since a pursuer pursuing a maneuvering target does not know what maneuvers an evading target will make, the maneuvers (the target's control law) appear as a random process to the pursuer. However, he has opinions about what the evader will do. From these, he can assign a prior probability distribution to the evader's maneuvers. For a linear pursuit evasion problem in which the evader's control law is modeled as a random process, in which the pursuer has partial noisy linear measurements of his own and the evader's relative position, and a quadratic optimality criterion is used, past results of the authors imply that the optimal control is a linear function of the “predicted miss”. Determining the predicted miss involves estimating the evader's terminal position from past system measurements. Nonlinear filtering techniques are used to give expressions for computing the conditional expectation of the evader's terminal position even in the presence of the random unknown maneuvers of the evader     

Simple Group Pursuit Subject to Phase Constraints and Data Delay

The problem of the simple pursuit of a single evader by a group of pursuers subject to phase constraints is considered. It is assumed that the pursuers receive information about the evader’s control and affect the control system with a time delay. Sufficient conditions for the solvability of the pursuit problem are obtained.     

多追捕者-单-逃跑者追逃问题实现成功捕获的约束条件

针对包含有n个追捕者及1个逃跑者的2维平面多机器人追逃问题,对实现成功捕获的约束条件进行了研究.经过理论分析得出:在机器人拥有全局视野的情况下,即使单一逃跑者性能优于每个追捕者,只要满足追捕者与逃跑者的速率比大于sin(π/n),逃跑机器人落在追捕机器人所构成的凸多边形内部且逃跑者和追捕者构成的相邻追-逃阿波罗尼奥斯圆满足两两相交(相切)这2个约束条件,则追捕者通过选择合适的追捕策略就一定可以实现成功抓捕.此外,还给出了在此约束条件下的追捕者和逃跑者的追逃策略.多组仿真实验同样证明了本文提出的约束条件是正确的.     

Policy Improvements for Probabilistic Pursuit-Evasion Game

This paper focuses on a pursuit-evasion game (PEG) which involves two teams: one side consists of pursuers trying to minimize the time required to capture evaders, and the other side consists of evaders trying to maximize the capture time by escaping the pursuers. In this paper, we propose a hybrid pursuit policy for a probabilistic PEG, which possesses the combined merits of local-max and global-max pursuit policies proposed in previous literature. A method to find optimal pursuit and evasion polices for two competitive parties of the pursuers and evaders is also proposed. For this, we employ an episodic parameter optimization (EPO) algorithm to learn good values for the weighting parameters of a hybrid pursuit policy and an intelligent evasion policy. The EPO algorithm is performed during the numerous repeated simulation runs of the PEG and the reward of each episode is updated using reinforcement learning, and the optimal weighting parameters are selected by using particle swarm optimization. We analyze the trend of the optimal parameter values with respect to the number of the pursuers and evaders. The proposed strategy is validated both in simulations and experiments with small ground robots.     

基于博弈论及Q学习的多Agent协作追捕算法

多Agent协作追捕问题是多Agent协调与协作研究中的一个典型问题。针对具有学习能力的单逃跑者追捕问题，提出了一种基于博弈论及Q学习的多Agent协作追捕算法。首先,建立协作追捕团队，并构建协作追捕的博弈模型；其次,通过对逃跑者策略选择的学习，建立逃跑者有限的Step-T累积奖赏的运动轨迹，并把运动轨迹调整到追捕者的策略集中；最后,求解协作追捕博弈得到Nash均衡解，每个Agent执行均衡策略完成追捕任务。同时,针对在求解中可能存在多个均衡解的问题，加入了虚拟行动行为选择算法来选择最优的均衡策略。C#仿真实验表明，所提算法能够有效地解决障碍环境中单个具有学习能力的逃跑者的追捕问题，实验数据对比分析表明该算法在同等条件下的追捕效率要优于纯博弈或纯学习的追捕算法。     

A cooperative pursuit-evasion game in wireless sensor and actor networks

This paper studies the problem of the pursuit-evasion game under the wireless sensor and actor networks (WSANs). In order to plan paths for pursuers to capture an evader in the pursuit-evasion game, a novel multi-step cooperative strategy is presented. Under this strategy, the pursuit-evasion game is studied in two stages. In the first stage we assume that the evader is always static in the workplace, and in the second stage the evader will move once it senses the existence of pursuers. A Daisy-Chain Formation algorithm and a sliding mode-based method are presented to control the pursuit. Based on Lyapunov stability theory, the proposed algorithm is proved to be convergent. At last, simulation results are provided to demonstrate the effectiveness of the proposed method.     

A Group Pursuit Game

A differential game of pursuit of an evader by m dynamic pursuers under simple motion is studied. The time of game completion is fixed. Pursuers’ controls obey integral constraints, whereas the evader control obeys either an integral constraint or a geometric constraint. A differential game with cost defined by the distance between the evader and his nearest pursuer at the game completion instant is studied. Optimal strategies for players are constructed and the game cost is determined.__________Translated from Avtomatika i Telemekhanika, No. 8, 2005, pp. 24–35.Original Russian Text Copyright © 2005 by Ibragimov.     

Pursuit evasion game with costly information

In this paper we study a pursuit evasion game in which information is costly. The pursuer has to pay, i.e. lose some time, whenever he wants information on the evader's position. Therefore the capture will be done in successive stages. The pursuer gets information, then moves using an open loop control, and so on. We characterize the setC ₁ of the initial states that the pursuer can capture in one stage whatever the evader does. This set is taken as a new target for an other stage. In this way we characterize the setC _n of the initial states the pursuer can capture inn stages in the worst case. We also give a pursuer's strategy that minimizes the total duration of the game, as opposed to the number of stages.This research has been supported in part by the French Direction des Recherches et Etudes Techniques (DRET).     

Scalable and practical pursuit-evasion with networked robots

In this paper, we consider the design and implementation of practical pursuit-evasion games with networked robots, where a communication network provides sensing-at-a-distance as well as a communication backbone that enables tighter coordination between pursuers. We first develop, using the theory of zero-sum games, an algorithm that computes the minimal completion time strategy for pursuit-evasion when pursuers and evaders have same speed, and when all players make optimal decisions based on complete knowledge. Then, we extend this algorithm to when evader are significantly faster than pursuers. Unfortunately, these algorithms do not scale beyond a small number of robots. To overcome this problem, we design and implement a partition algorithm where pursuers capture evaders by decomposing the game into multiple multi-pursuer single-evader games. We show that the partition algorithm terminates, has bounded capture time, is robust, and is scalable in the number of robots. We then describe the design of a real-world mobile robot-based pursuit evasion game. We validate our algorithms by experiments in a moderate-scale testbed in a challenging office environment. Overall, our work illustrates an innovative interplay between robotics and communication.     

A Decentralized Fuzzy Learning Algorithm for Pursuit-Evasion Differential Games with Superior Evaders

In this paper, we consider multi-pursuer single-superior-evader pursuit-evasion differential games where the evader has a speed that is similar to or higher than the speed of each pursuer. A new fuzzy reinforcement learning algorithm is proposed in this work. The proposed algorithm uses the well-known Apollonius circle mechanism to define the capture region of the learning pursuer based on its location and the location of the superior evader. The proposed algorithm uses the Apollonius circle with a developed formation control approach in the tuning mechanism of the fuzzy logic controller (FLC) of the learning pursuer so that one or some of the learning pursuers can capture the superior evader. The formation control mechanism used by the proposed algorithm guarantees that the pursuers are distributed around the superior evader in order to avoid collision between pursuers. The formation control mechanism used by the proposed algorithm also makes the Apollonius circles of each two adjacent pursuers intersect or be at least tangent to each other so that the capture of the superior evader can occur. The proposed algorithm is a decentralized algorithm as no communication among the pursuers is required. The only information the proposed algorithm requires is the position and the speed of the superior evader. The proposed algorithm is used to learn different multi-pursuer single-superior-evader pursuit-evasion differential games. The simulation results show the effectiveness of the proposed algorithm.     

An efficient algorithm for optimal routing applied to convoy merging manoeuvres in urban environments

This paper describes an innovative routing strategy for intelligent transportation units willing to perform merging manoeuvres with a moving convoy. In particular, we consider a transportation unit located inside a city (pursuer unit), and which wishes to join a convoy that is constantly moving around the city. We first describe a solution that considers idealistic conditions, i.e., the traveling time along each street is constant. We then go on to improve our first approach to deal with the realistic random nature of the traveling times experienced by the pursuer and by the convoy leader. Our search strategy applies Dynamic Programming to achieve a meeting point that is optimal in two ways: on one hand, the optimal destination is the one closest to the current pursuer??s position; on the other hand, the optimal meeting point must minimize the time that elapses until the pursuer meets the convoy (considering that the pursuer must always arrive first). Calculating the optimal path to every possible destination is highly inefficient, error prone and time consuming. On the contrary, we propose an efficient search strategy that will achieve the meeting point and the path to it, both optimally and in a very short time. This enables the real-time application of our approach either for finding new solutions or for re-planning old ones owing to unexpected real-conditions.