引入通信与探索的多智能体强化学习QMIX算法

非平稳性问题是多智能体环境中深度学习面临的主要挑战之一，它打破了大多数单智能体强化学习算法都遵循的马尔可夫假设，使每个智能体在学习过程中都有可能会陷入由其他智能体所创建的环境而导致无终止的循环。为解决上述问题，研究了中心式训练分布式执行（CTDE）架构在强化学习中的实现方法，并分别从智能体间通信和智能体探索这两个角度入手，采用通过方差控制的强化学习算法（VBC）并引入好奇心机制来改进QMIX算法。通过星际争霸Ⅱ学习环境（SC2LE）中的微操场景对所提算法加以验证。实验结果表明，与QMIX算法相比，所提算法的性能有所提升，并且能够得到收敛速度更快的训练模型。

Improved QMIX algorithm from communication and exploration for multi-agent reinforcement learning

Non-stationarity that breaks the Markov assumption followed by most single-agent reinforcement learning algorithms is one of the main challenges in multi-agent environment， making each agent may be caught in an infinite loop caused by the environment created by the other agents during the learning process. To solve above problem， the implementation method of Centralized Training with Decentralized Execution （CTDE） structure in reinforcement learning was studied， and from two perspectives of agent communication and exploration， the QMIX algorithm was improved by introducing a Variance Control-Based （VBC） communication model and a curiosity mechanism. The proposed algorithm was validated in micro control scenarios of StarCraft Ⅱ Learning Environment （SC2LE）. Experimental results show that the proposed algorithm can improve the performance and obtain a training model with higher convergence speed compared to QMIX algorithm.