融合认知行为模型的深度强化学习框架及算法

面对高维连续状态空间或稀疏奖励等复杂任务时,仅依靠深度强化学习算法从零学习最优策略十分困难,如何将已有知识表示为人与学习型智能体之间相互可理解的形式,并有效地加速策略收敛仍是一个难题.对此,提出一种融合认知行为模型的深度强化学习框架,将领域内先验知识建模为基于信念-愿望-意图(belief- desire-intention, BDI)的认知行为模型,用于引导智能体策略学习.基于此框架,分别提出融合认知行为模型的深度Q学习算法和近端策略优化算法,并定量化设计认知行为模型对智能体策略更新的引导方式.最后,通过典型gym环境和空战机动决策对抗环境,验证所提出算法可以高效利用认知行为模型加速策略学习,有效缓解状态空间巨大和环境奖励稀疏的影响.     

基于多智能体的作战模拟仿真模型框架研究

传统的作战模拟仿真模型大都是数学模型，难以刻画战争推进过程中所表现出来的复杂的战场态势变化及其内在的动力学机制。复杂适应系统理论和基于多智能体的建模仿真方法为作战模拟研究提供了新的思路。该文以复杂适应系统理论和基于多智能体的建模仿真方法为指导，提出了一种基于多智能体的作战模拟仿真模型的框架，着重分析了其中智能体的结构，对智能体的属性和行为进行了进一步的讨论。通过与面向对象的方法中对象概念的比较，认为面向对象的方法是实现基于多智能体的作战模拟仿真模型的有效途径。     

基于智能体的行人路径查找微观模型研究

已有的行人路径查找模型大多数只关心行人群体的行为,而对行人的个体行为和思维方式几乎没有研究。为了更能体现行人的个体行为、思维方式以及行人与环境的交互作用,本文采用智能体和分层路网的思想,首先对行人智能体所处的路网进行分类,其中对机动车路网采用路线轨迹法,对行人采用无轨迹的路网描述;其次赋予行人一些复杂的的规则,并加入吸引点效应、信号灯效应和行人的群效应等约束规则,分析行人与其它行人、机动车、路网以及信号灯等的交互模型,体现行人智能体的自主性、反应性和社会性等;最后初步描述行人智能体复杂的路径查找微观模型全过程。通过对比分析几种典型的行人路径查找模型,表明本文的行人路径查找微观模型更能反映现实生活中的行行人路径查找过程,更能体现行人的智能性。     

基于指令机制的BDC思维状态模型研究

在作战仿真中,需要仿真作战单元的各种行为过程,而其行为必须依靠其思维状态过程驱动,研究了基于BDI理论的Agent认知模型,并讨论了BDI模型在作战应用中存在的不足的基础上,提出了更适用于军事作战的以指令(Command)为中心的BDC思维状态模型,实现了用“指令”代替“意图”作为智能体思维状态的核心来讨论个体的思维特...     

坦克分队作战仿真系统中的坦克机动模型研究

根据坦克机动作战的一般原则,对影响坦克机动的因素进行了较为详细的描述,提出了坦克机动模型的基本框架,分析了坦克机动模型的运行流程,研究了坦克机动数学模型及其计算方法,并在此基础上,采用C+ +语言进行了实现.     

自行高炮CGF实体智能机动行为仿真

CGF系统是未来作战仿真所必备的技术支持,在介绍CGF系统的基础上,提出了以智能仿真为依托,兼顾底层机动自主和高层决策可控的设计思路。研究了自行高炮连CGF实体在机动性能方面的仿真,通过对高炮机动性能中物理行为和认知行为的研究,运用BP神经网络实现CGF实体智能机动行为的建模,并建立了动态障碍的规避模块。较好解决了中各车行进中出现的实体碰撞问题,结合实际应用对模型进行了可行性验证。     

作战模拟的人工生命多智能体建模

作战是典型的复杂自适应系统,传统的作战模拟依赖于兰彻斯特方程等数学模型,将作战视为确定性过程,难以对作战过程中的许多无形因素建模。人工生命的模拟方法,通过多智能体之间的相互作用研究系统的突现行为,很好地揭示了作战的本质和作战过程的演化规律。提出了多智能体作战模拟的二维网格环境下的智能体进化基因模型。通过计算和考察虚拟战场环境中的敌、我、友多方在交战过程中的演变规律和涌现特征,可模拟和解释战场局势的推演行为和发展规律。采用竞争、学习、协作进化、群居、物种灭绝等等生命进化现象来完成系统复杂性的模拟,取得了较好的仿真效果。模型系统具有演化速度快,模拟结果合理和可交互性等特点。     

一种综合可计算情感建模方法

情感作为人类具有智能的一个重要体现,是创建可信服的虚拟智能体不可或缺的环节.如何通过情感来提高虚拟智能体的智能性和可信服性已成为亟待解决的关键问题.结合生理和认知对情感的影响,提出了一种综合的可计算情感建模方法;设计了一个完全的过程框架,以描述情感在不同时刻如何动态变化以及如何处理多种混合情感的情况;并建立了基于具体描述事件和情感关系的情感结构,以产生具体、真实的情感行为;提出了交互学习机制以增强虚拟智能体对动态环境的适应能力.实例验证表明,此情感模型能有效地增加虚拟智能体在交互过程中的可信服性.     

舰艇智能兵力决策CBPN模型与实现研究

智能兵力作战决策支持是计算机生成兵力构建的重难点和核心内容,传统海战智能兵力作战决策系统难以满足决策知识空间增加对系统决策正确性和灵活性的要求,研究将Petri网建模方法与CXBR上下文推理方法相结合,提出了智能兵力CBPN图形化决策建模方法,依据舰艇智能兵力作战决策过程,分层构造了舰艇智能兵力作战行为模型,并建立了舰艇智能兵力决策支持的CBPN模型,实现了业务层和软件代码层解耦,最后构建了舰艇智能兵力的作战决策知识库.工程实现结果证明新方法能显著提高系统开发效率和智能兵力决策的灵活性,满足复杂作战仿真系统智能兵力作战决策支持需求.     

基于线性时序逻辑的多Agent协商推理模型

在多智能体系统中,协商是Agent交互的主要形式.用形式化方法构建了基于线性时序逻辑的协商推理模型,该模型用线性时序逻辑描述在协商过程中Agent所处环境,自身能力、权力、知识、思维等随时间的变化,以及在系统运行时Agent采取异步行为.进一步完善了多Agent系统中自主的协商机制.     

基于HLA的多代理环境

由于对智能代理应用需求的不断增长，代理间合作与相互通信变得越来越重要。代理很少孤立存在，它需要一个环境来作为其宿主以支持通信与信息发布。该文介绍了利用HLA／RTI作为多代理环境的优点，以及设计实现代理行为体系结构的方法。最后文章给出了一个海军战斗想定的实例。     

虚拟驾驶环境中车辆智能体的驾驶行为模型

建立虚拟交通环境的多智能体结构,分析车辆智能体的驾驶行为分层模型以及感知、决策和操作等过程。采用模糊专家系统建立车辆智能体的驾驶行为模型。为模拟现实中的驾驶员行为特性,加入驾驶员因子,使驾驶模拟器的虚拟交通环境更符合现实。运用OpenGVS产生和显示实时交互的虚拟驾驶场景。结果表明该模型能体现实际驾驶行为的多样性、随机性和模糊性。该模型通用有效,它使驾驶模拟器的虚拟交通场景更真实满意。     

A simplified combat model based on a cellular automaton

The work is devoted to constructing a cellular automaton based model of motion and combat activity of systems of hierarchically organized agents with allowance for terrain ruggedness, communication channels between the agents, and the necessity of choosing behavior adequate to the current situation.     

基于演化规划的多智能体系统中动态行为的建模

在多智能体系统中，每个智能体必须使自身适应环境动态地同其它智能体协调。为达到此目标，智能体须有预测其他智能体的行为及与其它智能体协作的能力，应动态地建立起自身的行为模型并且不断的演化它。     

多阶段建模过程中可重用数学模型表示方法

为解决多阶段建模过程中数学模型重用性问题,分析多阶段建模过程中数学模型表示现状及MDA(model driven architecture,模型驱动体系结构)对数学模型的重用性和平台无关性需求,提出作战训练仿真数学模型广义定义和面向重用的数学模型表示方法;通过建立可重用变量转换关系网络组织、管理和描述作战训练仿真数学模型中的变量转换关系(variable conversion relation,VCR),扩展MathML(math markup language,数学标记语言)语义并基于扩展的MathML语义对变量转换数学表达式进行表示。最后基于变量转换关系网络实现作战训练仿真数学模型动态逻辑行为模型生成。     

一种基于Agent的自适应软件过程模型

传统的软件过程模型大多是静态的、机械的、被动的,它们要求软件工程人员在描述软件过程时预期所有可能发生的情况,并且显式地定义这些问题的解决方案.当软件过程所处的环境发生变化时,软件过程无法自适应地对这些变更作出相应的调整.提出了一种基于Agent的自适应软件过程模型.在这种软件过程模型中,软件过程被描述为一组相互独立而对等的实体--软件过程Agent.这些软件过程Agent能够对软件过程环境的变化主动地、自治地作出反应,动态地确定和变更其行为以实现软件开发的目标.     

Design, development and validation of an agent-based model of electronic auctions

This paper presents the design, development and validation methodology of an agent-based computational model of the B2C electronic auction marketplace. It aims at a comprehensive understanding of the varied issues governing a B2C electronic auction, incorporating the behavior of all relevant agents such as the auctioneer, the consumer and the retailer; and the environment in which these agents operate and interact. In contrast with conventional methods, agent based modeling employs a bottom-up modeling approach where behaviors of individual agents and rules for their interaction, specified at the micro level, give rise to emergent macro level phenomenon. The development methodology should ensure that agent models are aligned with theory, current knowledge of the field and observed phenomena, and output validity of the model also needs to be ascertained. Beginning with a general introduction to agent based computational modeling, this paper formalizes this alignment and validation methodology and elaborates each step, noting the rationale and means for achieving these. The manner in which this process was used in modeling B2C auctions is then described.     

Conceptualization and implementation of a multi-agent model to simulate whale-watching tours in the St. Lawrence Estuary in Quebec, Canada

The Saguenay St. Lawrence Marine Park (SSLMP) and the adjacent Marine Protected Area (MPA) in the St. Lawrence Estuary, in Quebec, cover a territory of exceptional biodiversity including 12 species of marine mammals, nearly half of which are considered to be endangered species. Whale-watching trips and other human activities related to commercial shipping, tourism, and recreation generate very intensive traffic in the area, which pose cumulative threats to the marine wildlife. This study has been undertaken in collaboration with the Marine Park and the MPA managers to develop a multi-agent system (MAS) to investigate the interactions between the traffic and the marine mammals in the estuary. This paper describes the first prototype version of the proposed MAS model where the focus is on the whale-watching boats. It discusses the conceptual model with its principal components: the physical environment and the boat agents and whale entities, and the implementation of the model with the behavior rules of the agents. In this version of the MAS, the whale-watching boats are represented as cognitive agents while the whales are simple reactive entities. The prototype model was implemented in the agent-based modeling platform RePast. An index, the happiness factor (i.e., the ratio of whale observation time over the trip duration) was designed to measure how successful the boat agents are in achieving their goal. Simulations were run to assess different decision strategies of the boat agents and their impacts on the whales. Results show that cooperative behavior that involves a combination of innovator and imitator strategies yields a higher average happiness factor over non-cooperative, purely innovators, behavior. However, this cooperative behavior creates increased risk for the whale populations in the estuary.     

Evolution,Generality and Robustness of Emerged Surrounding Behavior in Continuous Predators-Prey Pursuit Problem

We present the result of our work on the use of strongly typed genetic programming with exception handling capabilities for the evolution of surrounding behavior of agents situated in an inherently cooperative environment. The predators-prey pursuit problem is used to verify our hypothesis that relatively complex surrounding behavior may emerge from simple, implicit, locally defined, and therefore—scalable interactions between the predator agents. Proposing two different communication mechanisms ((i) simple, basic mechanism of implicit interaction, and (ii) explicit communications among the predator agents) we present a comparative analysis of the implications of these communication mechanisms on evolution, generality and robustness of the emerged surrounding behavior. We demonstrate that relatively complex-surrounding behavior emerges even from implicit, proximity-defined interactions among the agents. Although the basic model offers the benefits of simplicity and scalability, compared to the enhanced model of explicit communications among the agents, it features increased computational effort and inferior generality and robustness of agents' emergent surrounding behavior when the team of predator agents is evolved in noiseless environment and then tested in noisy and uncertain environment. Evolution in noisy environment virtually equalizes the robustness and generality characteristics of both models. For both models however the increase of noise levels during the evolution is associated with evolving solutions, which are more robust to noise but less general to new, unknown initial situations.