• Login
    View Item 
    •   SMARTech Home
    • Georgia Tech Theses and Dissertations
    • Georgia Tech Theses and Dissertations
    • View Item
    •   SMARTech Home
    • Georgia Tech Theses and Dissertations
    • Georgia Tech Theses and Dissertations
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    Adaptable and Scalable Multi-Agent Graph-Attention Communication

    Thumbnail
    View/Open
    NIU-THESIS-2022.pdf (7.924Mb)
    Date
    2022-05-04
    Author
    Niu, Yaru
    Metadata
    Show full item record
    Abstract
    High-performing teams learn effective communication strategies to judiciously share information and reduce the cost of communication overhead. Within multi-agent reinforcement learning, synthesizing effective policies requires reasoning about when to communicate, whom to communicate with, and how to process messages. Meanwhile, in real-world problems, training policies and communication strategies that are able to generalize to multiple tasks, and adapt to unseen tasks, can improve the learning efficiency in multi-agent systems. However, many methods in current literature suffer from efficiently learning a dynamic communication topology. At the same time, learning adaptable and scalable multi-agent communication remains to be a challenge. This thesis develops algorithms to tackle these two problems. First, I propose a novel multi-agent reinforcement learning algorithm, Multi-Agent Graph-attention Communication (MAGIC), with a graph-attention communication protocol in which we learn 1) a Scheduler to help with the problems of when to communicate and whom to address messages to, and 2) a Message Processor using Graph Attention Networks (GATs) with dynamic graphs to deal with communication signals. The Scheduler consists of a graph attention encoder and a differentiable attention mechanism, which outputs dynamic, differentiable graphs to the Message Processor, which enables the Scheduler and Message Processor to be trained end-to-end. We evaluate our approach on a variety of cooperative tasks, including Google Research Football. Our method outperforms baselines across all domains, achieving approximately 10.5% increase in reward in the most challenging domain. We also show MAGIC communicates 27.4% more efficiently on average than baselines, is robust to stochasticity, and scales to larger state-action spaces. Finally, we demonstrate MAGIC on a physical, multi-robot testbed. Second, based on MAGIC, I present a multi-agent multi-task reinforcement training scheme, MT-MAGIC, and develop a multi-agent meta-reinforcement learning framework, Meta-MAGIC. Both methods can generalize and adapt to unseen tasks with different team sizes. Meta-MAGIC initiatively explores using the RNN architecture to perform the adaptation process in multi-agent meta-reinforcement learning. Through experiments, we find that Meta-MAGIC and MT-MAGIC can beat the baseline by a notable margin in multi-task training and generalize well to new tasks. Meta-MAGIC is able to adapt quickly to new tasks and keeps an upper bound of the performance of all methods through the interactions with unseen scenarios in Predator-Prey. Fine-tuning from pre-trained models by MT-MAGIC quickly achieves better performance on new tasks compared to training from scratch, with only 11.28% of training epochs.
    URI
    http://hdl.handle.net/1853/66659
    Collections
    • Georgia Tech Theses and Dissertations [23403]
    • School of Electrical and Computer Engineering Theses and Dissertations [3303]

    Related items

    Showing items related by title, author, creator and subject.

    • Randomized Multi-Source Broadcast Protocols in Multi-Hop Radio Networks 

      Lee, Chungki; Ammar, Mostafa H. (Mostafa Hamed); Burns, James Edward (Georgia Institute of Technology, 1993)
      In [LBA93], a suite of randomized broadcast protocols was presented for the problem of broadcasting a message in arbitrary multi-hop radio networks. These protocols improved upon that of Bar-Yehuda et al. in most of the ...
    • A Multi-physics approach to the co-design of 3D multi-core processors 

      Xiao, He (Georgia Institute of Technology, 2018-01-22)
      The three-dimensional integrated circuit (3D IC) is a promising solution for processors in the post-Moore era. The 3D integration stacks multiple dies vertically in a single package and enables high integration density. ...
    • Multi Perspectives on the Multi-Modal Passenger Terminal 

      Alhadeff, Heather; Calvert, Christopher M.; Cooper, Darren; Hatton, Tim; Howard, Herman H.; Phillips, Marion; Strickland, Katrina (Georgia Institute of Technology, 2013-11-07)
      A discussion about the Multi-Modal Passenger Terminal, a ground-breaking transportation project in downtown Atlanta from those directly involved in the different aspects of the project.

    Browse

    All of SMARTechCommunities & CollectionsDatesAuthorsTitlesSubjectsTypesThis CollectionDatesAuthorsTitlesSubjectsTypes

    My SMARTech

    Login

    Statistics

    View Usage StatisticsView Google Analytics Statistics
    facebook instagram twitter youtube
    • My Account
    • Contact us
    • Directory
    • Campus Map
    • Support/Give
    • Library Accessibility
      • About SMARTech
      • SMARTech Terms of Use
    Georgia Tech Library266 4th Street NW, Atlanta, GA 30332
    404.894.4500
    • Emergency Information
    • Legal and Privacy Information
    • Human Trafficking Notice
    • Accessibility
    • Accountability
    • Accreditation
    • Employment
    © 2020 Georgia Institute of Technology