Adaptable and Scalable Multi-Agent Graph-Attention Communication
MetadataShow full item record
High-performing teams learn effective communication strategies to judiciously share information and reduce the cost of communication overhead. Within multi-agent reinforcement learning, synthesizing effective policies requires reasoning about when to communicate, whom to communicate with, and how to process messages. Meanwhile, in real-world problems, training policies and communication strategies that are able to generalize to multiple tasks, and adapt to unseen tasks, can improve the learning efficiency in multi-agent systems. However, many methods in current literature suffer from efficiently learning a dynamic communication topology. At the same time, learning adaptable and scalable multi-agent communication remains to be a challenge. This thesis develops algorithms to tackle these two problems. First, I propose a novel multi-agent reinforcement learning algorithm, Multi-Agent Graph-attention Communication (MAGIC), with a graph-attention communication protocol in which we learn 1) a Scheduler to help with the problems of when to communicate and whom to address messages to, and 2) a Message Processor using Graph Attention Networks (GATs) with dynamic graphs to deal with communication signals. The Scheduler consists of a graph attention encoder and a differentiable attention mechanism, which outputs dynamic, differentiable graphs to the Message Processor, which enables the Scheduler and Message Processor to be trained end-to-end. We evaluate our approach on a variety of cooperative tasks, including Google Research Football. Our method outperforms baselines across all domains, achieving approximately 10.5% increase in reward in the most challenging domain. We also show MAGIC communicates 27.4% more efficiently on average than baselines, is robust to stochasticity, and scales to larger state-action spaces. Finally, we demonstrate MAGIC on a physical, multi-robot testbed. Second, based on MAGIC, I present a multi-agent multi-task reinforcement training scheme, MT-MAGIC, and develop a multi-agent meta-reinforcement learning framework, Meta-MAGIC. Both methods can generalize and adapt to unseen tasks with different team sizes. Meta-MAGIC initiatively explores using the RNN architecture to perform the adaptation process in multi-agent meta-reinforcement learning. Through experiments, we find that Meta-MAGIC and MT-MAGIC can beat the baseline by a notable margin in multi-task training and generalize well to new tasks. Meta-MAGIC is able to adapt quickly to new tasks and keeps an upper bound of the performance of all methods through the interactions with unseen scenarios in Predator-Prey. Fine-tuning from pre-trained models by MT-MAGIC quickly achieves better performance on new tasks compared to training from scratch, with only 11.28% of training epochs.
Showing items related by title, author, creator and subject.
Al Abri, Said Salim Hamdan S. (Georgia Institute of Technology, 2019-06-26)Modeling and control of multi-agent systems is an important problem due to its large variety of potential applications and increasing practical and theoretical challenges. A large part of inspiration for modeling and ...
Hendricks, Eric Scott (Georgia Institute of Technology, 2017-01-09)In order to address key challenges facing the aviation industry, NASA’s Advanced Air Vehicle Program researches new aircraft technologies and concepts. Developing these new technologies and concepts is becoming increasingly ...
An interference-cancellation receiver for multi-band and multi-standard wireless communication systems Beck, Sungho (Georgia Institute of Technology, 2011-06-24)The dissertation presents novel methodologies to realize a multi-band and multi-standard receiver with an interference-cancellation capability. First, the receiver specifications are derived from the wireless communication ...