When Does Communication Learning Need Hierarchical Multi-Agent Deep Reinforcement Learning |
| |
Authors: | Marie Ossenkopf Mackenzie Jorgensen Kurt Geihs |
| |
Affiliation: | 1. Distributed Systems Group, University of Kassel, Kassel, Germany;2. mos@vs.uni-kassel.de;4. Computing Sciences, Villanova University, Villanova, Pennsylvania, USA |
| |
Abstract: | AbstractMulti-agent systems need to communicate to coordinate a shared task. We show that a recurrent neural network (RNN) can learn a communication protocol for coordination, even if the actions to coordinate are performed steps after the communication phase. We show that a separation of tasks with different temporal scale is necessary for successful learning. We contribute a hierarchical deep reinforcement learning model for multi-agent systems that separates the communication and coordination task from the action picking through a hierarchical policy. We further on show, that a separation of concerns in communication is beneficial but not necessary. As a testbed, we propose the Dungeon Lever Game and we extend the Differentiable Inter-Agent Learning (DIAL) framework. We present and compare results from different model variations on the Dungeon Lever Game. |
| |
Keywords: | Agent communication deep reinforcement learning hierarchical learning multi-agent systems |
|
|