基于多智能体的Option自动生成算法

目前分层强化学习中的任务自动分层都是采用基于单智能体的串行学习算法，为解决串行算法学习速度较慢的问题，以Sutton的Option分层强化学习方法为基础框架，提出了一种基于多智能体的Option自动生成算法，该算法由多智能体合作对状态空间进行并行探测并集中应用aiNet实现免疫聚类产生状态子空间，然后并行学习生成各子空间上的内部策略，最终生成Option．以二维有障碍栅格空间内2点间最短路径规划为任务背景给出了算法并进行了仿真实验和分析．结果表明，基于多智能体的Option自动生成算法速度明显快于基于单智能体的算法。

Algorithm for automatic constructing Option based on multi-agent

In current hierarchical reinforcement learning, the automatic task hierarchies are constructed by low speed serial learning algorithm based on single-agent. A multi-agent based algorithm for constructing Options automatically was presented for speeding up the learning algorithm. The algorithm was developed on the basis of the Option HRL framework proposed by Sutton. Firstly, multiple agents cooperated in parallel exploring the state space. Then the state space was partitioned into several sub spaces via immune clustering based on aiNet. Next, the agents learned the local strategies of the different sub-space concurrently. Consequently, the Options were constructed. The theoretical analyses and experiments with shortest path planning in a two-dimensional grid space with obstacles show that the speed of muhi-agent based algorithm for automatically constructing Options was obviously faster than that of single-agent based algorithms.