Collaborative privacy preserving multi-agent planning

In many cases several entities, such as commercial companies, need to work together towards the achievement of joint goals, while hiding certain private information. To collaborate effectively, some sort of plan is needed to coordinate the different entities. We address the problem of automatically generating such a coordination plan while preserving the agents’ privacy. Maintaining privacy is challenging when planning for multiple agents, especially when tight collaboration is needed and a global high-level view of the plan is required. In this work we present the Greedy Privacy-Preserving Planner (GPPP), a privacy preserving planning algorithm in which the agents collaboratively generate an abstract and approximate global coordination plan and then individually extend the global plan to executable plans. To guide GPPP, we propose two domain independent privacy preserving heuristics based on landmarks and pattern databases, which are classical heuristics for single agent search. These heuristics, called privacy-preserving landmarks and privacy preserving PDBs, are agnostic to the planning algorithm and can be used by other privacy-preserving planning algorithms. Empirically, we demonstrate on benchmark domains the benefits of using these heuristics and the advantage of GPPP over existing privacy preserving planners for the multi-agent STRIPS formalism.     

Verifying plans for multiple agents

Abstract

Research in distributed artificial intelligence planning has historically focused on two distinct classes of problems. One paradigm has been that of 'planning for multiple agents', which considers issues inherent in centrally directed multi-agent execution. The second paradigm has been 'distributed planning', where multiple agents more autonomously participate in coordinating and deciding upon their own actions. The work described in this paper is in the first category, planning for multiple agents. Taking the STRIPS representation of actions, and directed acrylic graphs (DAGs) as plan representations particularly well suited to parallel execution, it formally analyses the following question: how can a DAG plan be verified (i.e. how can we be sure such a plan will be correct, given our uncertainty about exactly when unconstrained parallel actions will be performed)? A method is presented for verifying the correctness of plans for multiple agents, represented as DAGs. The technique allows for the efficient analysis of a plan, despite its many potential execution histories.     

Coordinating Self-interested Planning Agents

We consider planning problems where a number of non-cooperative agents have to work on a joint problem. Such problems consist in completing a set of interdependent, hierarchically ordered tasks. Each agent is assigned a subset of tasks to perform for which it has to construct a plan. Since the agents are non-cooperative, they insist on planning independently and do not want to revise their individual plans when the joint plan has to be assembled from the individual plans. We present a general formal framework to study some computational aspects of this non-cooperative coordination problem and we establish some complexity results to identify some of the factors that contribute to the complexity of this problem. Finally, we illustrate our approach with an application to coordination in multi-modal logistic planning.     

Action dependencies in privacy-preserving multi-agent planning

Collaborative privacy-preserving planning (CPPP) is a multi-agent planning task in which agents need to achieve a common set of goals without revealing certain private information. In many CPPP algorithms, the individual agents reason about a projection of the multi-agent problem onto a single-agent classical planning problem. For example, an agent can plan as if it controls the public actions of other agents, ignoring any private preconditions and effects theses actions may have, and use the cost of this plan as a heuristic estimate of the cost of the full, multi-agent plan. Using such a projection, however, ignores some dependencies between agents’ public actions. In particular, it does not contain dependencies between public actions of other agents caused by their private facts. We propose a projection in which these private dependencies are maintained. The benefit of our dependency-preserving projection is demonstrated by using it to produce high-level plans in a new privacy-preserving planner, and as a heuristic for guiding forward search privacy-preserving algorithms. Both are able to solve more benchmark problems than any other state-of-the-art privacy-preserving planner. This more informed projection does not explicitly expose any private fact, action, or precondition. In addition, we show that even if an adversary agent knows that an agent has some private objects of a given type (e.g., trucks), it cannot infer the number of such private objects that the agent controls. This introduces a novel form of strong privacy, which we call object-cardinality privacy, that is motivated by real-world requirements.     

Planning in a hierarchy of abstraction spaces

A problem domain can be represented as a hierarchy of abstraction spaces in which successively finer levels of detail are introduced. The problem solver ABSTRIPS, a modification of STRIPS, can define an abstraction space hierarchy from the STRIPS representation of a problem domain, and it can utilize the hierarchy in solving problems. Examples of the system's performance are presented that demonstrate the significant increases in problem-solving power that this approach provides. Then some further implications of the hierarchical planning approach are explored.     

FMAP: Distributed cooperative multi-agent planning

This paper proposes FMAP (Forward Multi-Agent Planning), a fully-distributed multi-agent planning method that integrates planning and coordination. Although FMAP is specifically aimed at solving problems that require cooperation among agents, the flexibility of the domain-independent planning model allows FMAP to tackle multi-agent planning tasks of any type. In FMAP, agents jointly explore the plan space by building up refinement plans through a complete and flexible forward-chaining partial-order planner. The search is guided by h _{D T G} , a novel heuristic function that is based on the concepts of Domain Transition Graph and frontier state and is optimized to evaluate plans in distributed environments. Agents in FMAP apply an advanced privacy model that allows them to adequately keep private information while communicating only the data of the refinement plans that is relevant to each of the participating agents. Experimental results show that FMAP is a general-purpose approach that efficiently solves tightly-coupled domains that have specialized agents and cooperative goals as well as loosely-coupled problems. Specifically, the empirical evaluation shows that FMAP outperforms current MAP systems at solving complex planning tasks that are adapted from the International Planning Competition benchmarks.     

Coordination by design and the price of autonomy

We consider a multi-agent planning problem as a set of activities that has to be planned by several autonomous agents. In general, due to the possible dependencies between the agents’ activities or interactions during execution of those activities, allowing agents to plan individually may lead to a very inefficient or even infeasible solution to the multi-agent planning problem. This is exactly where plan coordination methods come into play. In this paper, we aim at the development of coordination by design techniques that (i) let each agent construct its plan completely independent of the others while (ii) guaranteeing that the joint combination of their plans always is coordinated. The contribution of this paper is twofold. Firstly, instead of focusing only on the feasibility of the resulting plans, we will investigate the additional costs incurred by the coordination by design method, that means, we propose to take into account the price of autonomy: the ratio of the costs of a solution obtained by coordinating selfish agents versus the costs of an optimal solution. Secondly, we will point out that in general there exist at least two ways to achieve coordination by design: one called concurrent decomposition and the other sequential decomposition. We will briefly discuss the applicability of these two methods, and then illustrate them with two specific coordination problems: coordinating tasks and coordinating resource usage. We also investigate some aspects of the price of autonomy of these two coordination methods.     

一种多智能体系统任务竞争模型及算法研究

致力于解决多智能体系统中的任务分配问题，基于社会生活中的竞争现象提出了一种多智能体竞争模型，同时提出了解决多智能体任务分配的详细算法．文章引入博弈论来研究存在相互外部约束条件下的个体选择问题．为了克服求解纳什均衡点的复杂性，本文采用了一步纳什均衡的方法．仿真结果证明了本模型的合理性和算法的有效性．     

Incremental planning using conceptual graphs

Abstract

When performing a planning or design task in many domains it is often difficult to specify in advance what the precise goals are. It is therefore useful to have a system in which the planning process is performed interactively, with the solution approaching the users' intent incrementally through iterations of the planning process. A planning system intended to function in this way must be able to take goal specifications interactively rather than all at once at the beginning of the planning process. The planning process then becomes one of satisfying new goals as they are given by the user, modifying as little as possible the results of previous planning work. Incremental planning is an approach to interactive planning problems that allows a system to create a plan incrementally, modifying a previous plan to satisfy new or more precise goal specifications. In this paper we present an incremental planning system called the general constraint system (GCS) that is based on the conceptual programming environment (CP) developed at New Mexico State University and we show an example of the use of the system for a simple civil engineering design problem     

New directions in distributed Nash equilibrium seeking based on monotone operator theory

There has been an increasing research interest in modeling, optimization and control of various multi-agent networks that have wide applications in industry, defense, security, and social areas, such as computing clusters, interconnected micro-grid systems, unmanned vessel swarms \cite{ChenJie}, power systems\cite{MeiS}, multiple UAV systems\cite{KolaricP} and sensor networks\cite{LiuR}. For non-cooperative agents that only concern selfish profit-maximizing, the decision making problem can be modelled and solved with the help of game theory, while Nash equilibrium (NE) seeking is at the core to solve the non-cooperative multi-agent games \cite{HenrikSandberg, IsraelAlvarez, Jong-ShiPang}. Distributed NE seeking methods are appealing compared with the center-based methods in large-scale networks due to its scalability, privacy protection, and adaptability. Recently, monotone operator theory is explored for distributed NE seeking, which is shown to provide an uniform framework for various algorithms in different scenarios. It has been gradually developing into a cutting-edge research field, with the prospect and necessity of future in-depth research. In non-cooperative multi-agent games, each agent has different characteristics and pursues maximizing its own benefit. Hence, there is no centralized manager that can force all agents to adopt specified strategies to optimize the overall benefits. Under the NE, no player can decrease its cost by unilaterally changing its local decision to another feasible one. To seek an NE, the agent is required to optimize its own objective function given the opponent''s countermeasures. Therefore, various optimization-based methods have been investigated for distributed NE seeking, such as the gradient flow method and the best response method....     

Game theoretical applications for multi-agent systems

We consider game-theoretic principles for design of cooperative and competitive (non-cooperative self-interested) multi-agent systems. Using economic concepts of tâtonnement and economic core, we show that cooperative multi-agent systems should be designed in games with dominant strategies that may lead to social dilemmas. Non-cooperative multi-agent systems, on the other hand, should be designed for games with no clear dominant strategies and high degree of problem complexity. Further, for non-cooperative multi-agent systems, the number of learning agents should be carefully managed so that solutions in the economic core can be obtained. We provide experimental results for the design of cooperative and non-cooperative MAS from telecommunication and manufacturing industries.     

Conflict resolution in collaborative planning dialogs

In a collaborative planning environment in which the agents are autonomous and heterogeneous, it is inevitable that discrepancies in the agents' beliefs result in conflicts during the planning process. In such cases, it is important that the agents engage in collaborative negotiation to resolve the detected conflicts in order to determine what should constitute their shared plan of actions and shared beliefs. This paper presents a plan-based model for conflict detection and resolution in collaborative planning dialogs. Our model specifies how a collaborative system should detect conflicts that arise between the system and its user during the planning process. If the detected conflicts warrant resolution, our model initiates collaborative negotiation in an attempt to resolve the conflicts in the agent's beliefs. In addition, when multiple conflicts arise, our model identifies and addresses the most effective aspect in its pursuit of conflict resolution. Furthermore, by capturing the collaborative planning process in a recursive Propose–Evaluate–Modify cycle of actions, our model is capable of handling embedded negotiation during conflict resolution.     

Agent能力承诺协作的自适应图规划协议生成算法

面向开放多智能体系统OMAS的自适应性体现在多Agent通过协作在运行时调整系统行为以适应环境的变化。针对现有多Agent设计时预定义协作机制无法满足运行时自适应协作的问题,提出了一种基于目标-能力-承诺GCC元模型生成自适应多Agent能力承诺协作图规划协议CCGP的算法。首先提出支持上下文环境语义相似度计算的GCC模型;然后将目标与能力（或者承诺）间的语义匹配度引入到基于能力协作的图规划协议中生成优化算法;最后以医疗垃圾AGV运输仿真系统为实验场景进行2组对比实验,实验结果表明CCGP算法对于生成图规划协议的执行时间有明显提高。     

An abstract knowledge based approach to diagnosis and recovery of plan failure in multi-agent systems

Open multi-agent systems (MAS) are decentralised and distributed systems that consist of a large number of loosely coupled autonomous agents. In the absence of centralised control they tend to be difficult to manage, especially in an open environment, which is dynamic, complex, distributed and unpredictable. This dynamism and uncertainty in an open environment gives rise to unexpected plan failures. In this paper we present an abstract knowledge based approach for the diagnosis and recovery of plan action failures. Our approach associates a sentinel agent with each problem solving agent in order to monitor the problem solving agent’s interactions. The proposed approach also requires the problem solving agents to be able to report on the status of a plan’s actions.Once an exception is detected the sentinel agents start an investigation of the suspected agents. The sentinel agents collect information about the status of failed plan abstract actions and knowledge about agents’ mental attitudes regarding any failed plan. The sentinel agent then uses this abstract knowledge and the agents’ mental attitudes, to diagnose the underlying cause of the plan failure. The sentinel agent may ask the problem solving agent to retry their failed plan based on the diagnostic result.     

Group planning with time constraints

Embedding planning systems in real-world domains has led to the necessity of Distributed Continual Planning (DCP) systems where planning activities are distributed across multiple agents and plan generation may occur concurrently with plan execution. A key challenge in DCP systems is how to coordinate activities for a group of planning agents. This problem is compounded when these agents are situated in a real-world dynamic domain where the agents often encounter differing, incomplete, and possibly inconsistent views of their environment. To date, DCP systems have only focused on cases where agents’ behavior is designed to optimize a global plan. In contrast, this paper presents a temporal reasoning mechanism for self-interested planning agents. To do so, we model agents’ behavior based on the Belief-Desire-Intention (BDI) theoretical model of cooperation, while modeling dynamic joint plans with group time constraints through creating hierarchical abstraction plans integrated with temporal constraints network. The contribution of this paper is threefold: (i) the BDI model specifies a behavior for self interested agents working in a group, permitting an individual agent to schedule its activities in an autonomous fashion, while taking into consideration temporal constraints of its group members; (ii) abstract plans allow the group to plan a joint action without explicitly describing all possible states in advance, making it possible to reduce the number of states which need to be considered in a BDI-based approach; and (iii) a temporal constraints network enables each agent to reason by itself about the best time for scheduling activities, making it possible to reduce coordination messages among a group. The mechanism ensures temporal consistency of a cooperative plan, enables the interleaving of planning and execution at both individual and group levels. We report on how the mechanism was implemented within a commercial training and simulation application, and present empirical evidence of its effectiveness in real-life scenarios and in reducing communication to coordinate group members’ activities.     

Primary and secondary diagnosis of multi-agent plan execution

Diagnosis of plan failures is an important subject in both single- and multi-agent planning. Plan diagnosis can be used to deal with plan failures in three ways: (i) to provide information necessary for the adjustment of the current plan or for the development of a new plan, (ii) to point out which equipment and/or agents should be repaired or adjusted to avoid further violation of the plan execution, and (iii) to identify the agents responsible for plan-execution failures. We introduce two general types of plan diagnosis: primary plan diagnosis identifying the incorrect or failed execution of actions, and secondary plan diagnosis that identifies the underlying causes of the faulty actions. Furthermore, three special cases of secondary plan diagnosis are distinguished, namely agent diagnosis, equipment diagnosis and environment diagnosis.     

基于后悔值的多Agent冲突博弈强化学习模型

对于冲突博弈,研究了一种理性保守的行为选择方法,即最小化最坏情况下Agent的后悔值.在该方法下,Agent当前的行为策略在未来可能造成的损失最小,并且在没有任何其他Agent信息的条件下,能够得到Nash均衡混合策略.基于后悔值提出了多Agent复杂环境下冲突博弈的强化学习模型以及算法实现.该模型中通过引入交叉熵距离建立信念更新过程,进一步优化了冲突博弈时的行为选择策略.基于Markov重复博弈模型验证了算法的收敛性,分析了信念与最优策略的关系.此外,与MMDP(multi-agent markov decision process)下Q学习扩展算法相比,该算法在很大程度上减少了冲突发生的次数,增强了Agent行为的协调性,并且提高了系统的性能,有利于维持系统的稳定.     

Adaptive selection of reactive/deliberate planning for a dynamic environment

This paper proposes and evaluates a new real-time reactive planning approach for a dynamic environment. In addition to having the features of conventional real-time reactive planning, which can react in a dynamic environment, our planning can perform deliberate planning appropriately. The proposed planning uses three kinds of agents: behavior agents that control simple behavior, planning agents that make plans to achieve their own goals, and behavior-selection agents that intermediate between behavior agents and planning agents. They coordinate a plan in an emergent way for the planning system as a whole. We confirmed the effectiveness of our planning by means of a simulation. Furthermore, we implemented an active-vision system and used it to verify the real-world efficiency of our planning.     

Hierarchical multi-agent planning for flexible assembly of large-scale lunar facilities

Several national space agencies and commercial aerospace companies plan to set up lunar bases with large-scale facilities that rely on multiple lunar robots’ assembly. Mission planning is necessary to achieve efficient multi-robot cooperation. This paper aims at autonomous multi-robot planning for the flexible assembly of the large-scale lunar facility, considering the harsh lunar environment, mission time optimization, and joint actions. The lunar robots and modules are scattered around the mission area without fixed assembly lines. Thus, the traditional assembly planning methods ignoring the optimal selection of modules are unable to handle this problem. We propose a hierarchical multi-agent planning method based on two-stage two-sided matching (HMAP-TTM) to solve this critical problem. First, the distributed planning framework with multi-replica public agents is introduced, ensuring robot plan knowledge consistency through public agents’ communication. Second, the hierarchical task graph (HTG) divides the mission into task layers based on task dependency knowledge. Third, we develop a novel two-stage two-sided matching algorithm. Time-optimal plans emerge from the matching games among public and private agents in each layer of HTG. Agents make decisions in the game based on action knowledge updated during planning. Finally, an assembly mission is presented to prove the method’s effectiveness. The simulation results show that the HMAP-TTM can generate plans with shorter mission time and require smaller communication costs than the baseline methods.