Multi-agent Physical A* with Large Pheromones

Physical A* (PHA*) and its multi-agent version MAPHA* are algorithms that find the shortest path between two points in an unknown real physical environment with one or many mobile agents [A. Felner et al. Journal of Artificial Intelligence Research, 21:631–679, 2004; A. Felner et al. Proceedings of the First International Joint Conference on Autonomous Agents and Multi-Agent Systems, Bologna, Italy, 2002:240–247]. Previous work assumed a complete sharing of knowledge between agents. Here we apply this algorithm to a more restricted model of communication which we call large pheromones, where agents communicate by writing and reading data at nodes of the graph that constitutes their environment. Previous works on pheromones usually assumed that only a limited amount of data can be written at each node. The large pheromones model assumes no limitation on the size of the pheromones and thus each agent can write its entire knowledge at a node. We show that with this model of communication the behavior of a multi-agent system is almost as good as with complete knowledge sharing. Under this model we also introduce a new type of agent, a communication agent, that is responsible for spreading the knowledge among other agents by moving around the graph and copying pheromones. Experimental results show that the contribution of communication agents is rather limited as data is already spread to other agents very well with large pheromones     

基于演化规划的多智能体系统中动态行为的建模

在多智能体系统中，每个智能体必须使自身适应环境动态地同其它智能体协调。为达到此目标，智能体须有预测其他智能体的行为及与其它智能体协作的能力，应动态地建立起自身的行为模型并且不断的演化它。     

不同支付函数形式对MG模型行为的影响

MG是描述多个体竞争性系统的一个结构简单但行为复杂的模型。MG模型的一个重要的机制是支付函数,它决定了对模型主体的财富和主体的策略的更新规则。该文研究了不同支付函数形式对模型的影响,发现支付函数只要是奖励赢方(少数方),惩罚输方(多数方),模型的突现行为就不会改变,与奖惩的具体形式无关,同时还发现模型突现行为有两个必要条件,一是主体必须竞争有限资源,二是在竞争中必须规定“拥挤效应”。     

一种基于社会承诺的Agent组织模型OMSC

多Agent领域所面临的一个重大的挑战是解决开放异质的多Agent系统中自治Agent间的协调问题。多Agent为了协调它们之间的活动,需要进行交互。社会承诺作为一种通信和交互机制,为自治的多Agent提供了一种协调的途径。然而,仅靠交互难以实现多Agent间的协调。Agent组织作为一种协调模型可以有效地控制多Agent间的交互与合作。论文将社会承诺和Agent组织两种协调机制相结合,提出一种基于社会承诺的Agent组织模型OMSC,分析了Agent如何用社会承诺进行推理以及基于社会承诺的多Agent系统并给出了一个实例,为多Agent间的协调提供了一种新的方法。     

Hierarchical multi-agent reinforcement learning

In this paper, we investigate the use of hierarchical reinforcement learning (HRL) to speed up the acquisition of cooperative multi-agent tasks. We introduce a hierarchical multi-agent reinforcement learning (RL) framework, and propose a hierarchical multi-agent RL algorithm called Cooperative HRL. In this framework, agents are cooperative and homogeneous (use the same task decomposition). Learning is decentralized, with each agent learning three interrelated skills: how to perform each individual subtask, the order in which to carry them out, and how to coordinate with other agents. We define cooperative subtasks to be those subtasks in which coordination among agents significantly improves the performance of the overall task. Those levels of the hierarchy which include cooperative subtasks are called cooperation levels. A fundamental property of the proposed approach is that it allows agents to learn coordination faster by sharing information at the level of cooperative subtasks, rather than attempting to learn coordination at the level of primitive actions. We study the empirical performance of the Cooperative HRL algorithm using two testbeds: a simulated two-robot trash collection task, and a larger four-agent automated guided vehicle (AGV) scheduling problem. We compare the performance and speed of Cooperative HRL with other learning algorithms, as well as several well-known industrial AGV heuristics. We also address the issue of rational communication behavior among autonomous agents in this paper. The goal is for agents to learn both action and communication policies that together optimize the task given a communication cost. We extend the multi-agent HRL framework to include communication decisions and propose a cooperative multi-agent HRL algorithm called COM-Cooperative HRL. In this algorithm, we add a communication level to the hierarchical decomposition of the problem below each cooperation level. Before an agent makes a decision at a cooperative subtask, it decides if it is worthwhile to perform a communication action. A communication action has a certain cost and provides the agent with the actions selected by the other agents at a cooperation level. We demonstrate the efficiency of the COM-Cooperative HRL algorithm as well as the relation between the communication cost and the learned communication policy using a multi-agent taxi problem.     

Discrete-event modeling of multi-agent systems with broadcasting-based parallel composition

Multi-agent systems have been widely used in logistics and manufacturing. In this paper we develop an automaton-based modeling framework for a special type of multi-agent systems, where agents are instantiated from a finite number of finite-state automaton templates, and interactions among agents are characterized via cooperative synchronization and broadcasting. To describe the compositional behavior of all agents, we propose a novel broadcasting-based parallel composition rule and show that it is commutative and associative. The effectiveness of this modeling framework and the parallel composition rule is illustrated in a simple multi-agent system.     

Coordinating Agents in Organizations Using Social Commitments

One of the main challenges faced by the multi-agent community is to ensure the coordination of autonomous agents in open heterogeneous multi-agent systems. In order to coordinate their behaviour, the agents should be able to interact with each other. Social commitments have been used in recent years as an answer to the challenges of enabling heterogeneous agents to communicate and interact successfully. However, coordinating agents only by means of interaction models is difficult in open multi-agent systems, where possibly malevolent agents can enter at any time and violate the interaction rules. Agent organizations, institutions and normative systems have been used to control the way agents interact and behave. In this paper we try to bring together the two models of coordinating agents: commitment-based interaction and organizations. To this aim we describe how one can use social commitments to represent the expected behaviour of an agent playing a role in an organization. We thus make a first step towards a unified model of coordination in multi-agent systems: a definition of the expected behaviour of an agent using social commitments in both organizational and non-organizational contexts.     

A novel approach to accelerate the convergence speed of a stochastic multi-agent system using recurrent neural nets

One problem in the design of multi-agent systems is the difficulty of predicting the occurrences that one agent might face, also to recognize and to predict their optimum behavior in these situations. Therefore, one of the most important characteristic of the agent is their ability during adoption, to learn, and correct their behavior. With consideration of the continuously changing environment, the back and forth learning of the agents, the inability to see the agent’s action first hand, and their chosen strategies, learning in a multi-agent environment can be very complex. On the one hand, with recognition to the current learning models that are used in deterministic environment that behaves linearly, which contain weaknesses; therefore, the current learning models are unproductive in complex environments that the actions of agents are stochastic. Therefore, it is necessary for the creation of learning models that are effective in stochastic environments. Purpose of this research is the creation of such a learning model. For this reason, the Hopfield and Boltzmann learning algorithms are used. In order to demonstrate the performance of their algorithms, first, an unlearned multi-agent model is created. During the interactions of the agents, they try to increase their knowledge to reach a specific value. The predicated index is the number of changed states needed to reach the convergence. Then, the learned multi-agent model is created with the Hopfield learning algorithm, and in the end, the learned multi-agent model is created with the Boltzmann learning algorithm. After analyzing the obtained figures, a conclusion can be made that when learning impose to multi-agent environment the average number of changed states needed to reach the convergence decreased and the use of Boltzmann learning algorithm decreased the average number of changed states even further in comparison with Hopfield learning algorithm due to the increase in the number of choices in each situation. Therefore, it is possible to say that the multi-agent systems behave stochastically, the more closer they behave to their true character, the speed of reaching the global solution increases.     

ALAN: adaptive learning for multi-agent navigation

In multi-agent navigation, agents need to move towards their goal locations while avoiding collisions with other agents and obstacles, often without communication. Existing methods compute motions that are locally optimal but do not account for the aggregated motions of all agents, producing inefficient global behavior especially when agents move in a crowded space. In this work, we develop a method that allows agents to dynamically adapt their behavior to their local conditions. We formulate the multi-agent navigation problem as an action-selection problem and propose an approach, ALAN, that allows agents to compute time-efficient and collision-free motions. ALAN is highly scalable because each agent makes its own decisions on how to move, using a set of velocities optimized for a variety of navigation tasks. Experimental results show that agents using ALAN, in general, reach their destinations faster than using ORCA, a state-of-the-art collision avoidance framework, and two other navigation models.     

ADK: An Agent Development Kit Based on a Formal Design Model for Multi-Agent Systems

The advent of multi-agent systems has brought us opportunities for the development of complex software that will serve as the infrastructure for advanced distributed applications. During the past decade, there have been many agent architectures proposed for implementing agent-based systems, and also a few efforts to formally specify agent behaviors. However, research on narrowing the gap between agent formal models and agent implementation is rare. In this paper, we propose a model-based approach to designing and implementing intelligent agents for multi-agent systems (MAS). Instead of using formal methods for the purpose of specifying agent behavior, we bring formal methods into the design phase of the agent development life cycle. Specifically, we use the formalism called agent-oriented G-net model, which is based on the G-net formalism (a type of high-level Petri net), to serve as the high-level design for intelligent agents. Based on the high-level design, we further derived the agent architecture and the detailed design for agent implementation. To demonstrate the feasibility of our approach, we developed the toolkit called ADK (Agent Development Kit) that supports rapid development of intelligent agents for multi-agent systems and we discuss the role of inheritance in agent-oriented development. As a potential solution for automated software development, we summarize the procedure to generate a model-based design of application-specific agents. Finally, to illustrate an application built on ADK, we present an air-ticket trading example.     

Predicting the Expected Behavior of Agents that Learn About Agents: The CLRI Framework

We describe a framework and equations used to model and predict the behavior of multi-agent systems (MASs) with learning agents. A difference equation is used for calculating the progression of an agent's error in its decision function, thereby telling us how the agent is expected to fare in the MAS. The equation relies on parameters which capture the agent's learning abilities, such as its change rate, learning rate and retention rate, as well as relevant aspects of the MAS such as the impact that agents have on each other. We validate the framework with experimental results using reinforcement learning agents in a market system, as well as with other experimental results gathered from the AI literature. Finally, we use PAC-theory to show how to calculate bounds on the values of the learning parameters.     

Learning collaboration strategies for committees of learning agents

A main issue in cooperation in multi-agent systems is how an agent decides in which situations is better to cooperate with other agents, and with which agents does the agent cooperate. Specifically in this paper we focus on multi-agent systems composed of learning agents, where the goal of the agents is to achieve a high accuracy on predicting the correct solution of the problems they encounter. For that purpose, when encountering a new problem each agent has to decide whether to solve it individually or to ask other agents for collaboration. We will see that learning agents can collaborate forming committees in order to improve performance. Moreover, in this paper we will present a proactive learning approach that will allow the agents to learn when to convene a committee and with which agents to invite to join the committee. Our experiments show that learning results in smaller committees while maintaining (and sometimes improving) the problem solving accuracy than forming committees composed of all agents.     

Prominence convergence in the collective synchronization of situated multi-agents

In previous work on collective synchronization of multi-agents, they always follow the assumptions that the synchronization is flat where all agents have the same synchronization capacity and the final synchronization result always converges on a common average strategy. However, in many circumstances the above assumption does not match the peculiarities of real multi-agent societies where each agent plays a different role in the synchronization. To make up the restrictions of related work, this paper presents a non-flat synchronization model where the synchronization capacity of each agent is different regarding its social rank and strategy dominance. In the presented model, all agents are situated in a synchronization field where each agent can sense the collective synchronization forces from other agents; if some agents are more prominent than other ordinary agents (e.g., they have the dominance of social ranks or behavior strategies), they will have strong synchronization capacities in the field; and finally the collective synchronization results may incline to converge at such prominent agents' strategies, which is called prominence convergence in collective synchronization and can be proved by our theoretical analyses and experimental results.     

Allocating training instances to learning agents for team formation

Agents can learn to improve their coordination with their teammates and increase team performance. There are finite training instances, where each training instance is an opportunity for the learning agents to improve their coordination. In this article, we focus on allocating training instances to learning agent pairs, i.e., pairs that improve coordination with each other, with the goal of team formation. Agents learn at different rates, and hence, the allocation of training instances affects the performance of the team formed. We build upon previous work on the Synergy Graph model, that is learned completely from data and represents agents’ capabilities and compatibility in a multi-agent team. We formally define the learning agents team formation problem, and compare it with the multi-armed bandit problem. We consider learning agent pairs that improve linearly and geometrically, i.e., the marginal improvement decreases by a constant factor. We contribute algorithms that allocate the training instances, and compare against algorithms from the multi-armed bandit problem. In our simulations, we demonstrate that our algorithms perform similarly to the bandit algorithms in the linear case, and outperform them in the geometric case. Further, we apply our model and algorithms to a multi-agent foraging problem, thus demonstrating the efficacy of our algorithms in general multi-agent problems.     

多主体系统中的信任管理

本文在前人工作的基础上提出一种改进方案来实现多主体系统的信任管理，即在多主体系统中建立安全服务器。安全服务器实现了基于X．509v3证书的主体身份鉴别，并根据主体X．509v3证书及相对应的属性证书和访问控制策略进行访问权限验证。安全服务器除实现目前广泛采用的访问控制类型之外，还实现了访问权限委派和委派链等访问控制
制类型。     

Broadcast stochastic receding horizon control of multi-agent systems

Optimal regulation of stochastically behaving agents is essential to achieve a robust aggregate behavior in a swarm of agents. How optimally these behaviors are controlled leads to the problem of designing optimal control architectures. In this paper, we propose a novel broadcast stochastic receding horizon control architecture as an optimal strategy for stabilizing a swarm of stochastically behaving agents. The goal is to design, at each time step, an optimal control law in the receding horizon control framework using collective system behavior as the only available feedback information and broadcast it to all agents to achieve the desired system behavior. Using probabilistic tools, a conditional expectation based predictive model is derived to represent the ensemble behavior of a swarm of independently behaving agents with multi-state transitions. A stochastic finite receding horizon control problem is formulated to stabilize the aggregate behavior of agents. Analytical and simulation results are presented for a two-state multi-agent system. Stability of the closed-loop system is guaranteed using the supermartingale theory. Almost sure (with probability 1) convergence of the closed-loop system to the desired target is ensured. Finally, conclusions are presented.     

Using agent- and organisation-oriented programming to develop a team of agents for a competitive game

The multi-agent programming contest uses a cow-herding scenario where two teams of cooperative agents compete for resources against each other. We developed such a team of agents using two well-known platforms, one based on a logic-based agent-oriented programming language, called Jason, and the other based on an organisational model, called $\mathcal{M}$ oise. While there is significant research on both agent programming and agent organisations, this was one of the first applications of a combined approach where we can program deliberative agents and organise them using a sophisticated organisational model. In this paper, we describe and discuss our contribution to the multi-agent contest using this combination of agent and organisation programming.     

Attribute-based authentication for multi-agent systems with dynamic groups

In a multi-agent system, agents are required to interact in order to exchange information. To achieve a reliable information exchange, a sound security protection must be in place. Unfortunately, security and privacy in multi-agent systems have not drawn adequate attention. They have been actually ignored or mistreated in most proposed multi-agent protocols. We observe that security and privacy issues are indeed not trivial and cannot be resolved with traditional security mechanisms, if agents are not trusted each other and their privacy must be protected. In this paper, we propose a secure multi-agent protocol that captures several most important security properties including agent privacy, data confidentiality, and agent authenticity. Intuitionally, we allow each agent in a group to hold a set of policy attributes. To access a protected data set, an agent must hold a correct policy attribute. In other words, the private information between two agents can be exchanged, if and only if the policy attribute embedded in the transmitted message matches that held by the receiver. In case of mismatching attributes, the private information of the corresponding agent will not be revealed to their counterpart. The proposed scheme is formalized with a sound cryptographic algorithm with a rigorous security proof.     

事件触发机制下的二阶多智能体系统的一致性

研究二阶多智能体系统的一致性问题.为了减少智能体之间的信息通信量,给出一种改进的事件触发控制方法, 在该方法下,每个智能体仅在自身事件触发时刻执行控制任务.利用模型转化、线性矩阵不等式方法和Lyapunov稳定性理论给出系统达到一致性的充分条件,同时,理论计算结果表明,系统在所提出的方法下不存在Zeno现象.仿真实例验证了理论分析的有效性.     

A first-order coalition logic for BDI-agents

Coalition logic (CL) enables us to model the strategic abilities and specify what a group of agents can achieve whatever the other agents do. However, some rational mental attitudes of the agents are beyond the scope of CL such as the prestigious beliefs, desires and intentions (BDI) which is an interesting and useful epistemic notion and has spawned substantial amount of studies in multi-agent systems. In this paper, we introduce a first-order coalition BDI (FCBDI) logic for multi-agent systems, which provides a semantic glue that allows the formal embedding and interaction of BDI, coalition and temporal operators in a first-order language. We further introduce a semantic model based on the interpreted system model and present an axiomatic system that is proved sound and complete with respect to the semantics. Finally, it is shown that the computational complexity of its model checking in finite structures is PSPACE-complete.