Date of Graduation
Fall 2024
Degree
Master of Science in Computer Science
Department
Computer Science
Committee Chair
Siming Liu
Abstract
Multi-Agent Reinforcement Learning (MARL) addresses complex tasks involving cooperation and competition among agents, training them to develop optimal policies for collective goals. However, facilitating simultaneous learning for multiple agents is challenging because the complexity increases rapidly with the number of agents. Current methods encompass a variety of centralized, decentralized, semi-centralized, and hybrid approaches to balance the trade-offs between computational efficiency, scalability, and coordination in MARL. In this study, I employed a centralized training with semi-centralized execution (CTSCE) framework, utilize both local observations from agents and abstracted global observations to effectively train agents in cooperative environments. Additionally, earning complex, domain-specific tasks from scratch in multi-agent systems is computationally expensive and time-consuming. Utilizing prior experiences can speed up the multi-agent reinforcement learning process. Previous work in our research lab has enhanced transfer learning for MARL by merging different state spaces into fixed-size inputs, enabling a single unified deep-learning policy to be applied across multiple scenarios within the popular MARL research platform, StarCraft Multi-Agent Challenge (SMAC). In this research, I enhance SMAC to Multi-Player enabled SMAC (MP-SMAC) by enabling the dynamic selection of training opponents. Utilizing MP-SMAC, I developed a co-evolving MARL framework, that establishes a co-evolutionary arena where multiple policies learn simultaneously. The arena involved the concurrent training of multiple policies across various scenarios, where they were tested against both static AI opponents and other policies within MP-SMAC. Additionally, I incorporated co-evolution with curriculum transfer learning into Co-MACTRL framework, enabling the MARL policies to systematically acquire knowledge and skills across organized scenarios of increasing difficulty, including adaptive opponents. The results showed significant improvements in MARL learning performance, highlighting the benefits of using co-evolving opponents and skills developed from diverse scenarios. Furthermore, Co-MACTRL learners consistently achieved high performance across a variety of SMAC scenarios, demonstrating the robustness and adaptability of the framework.
Keywords
Deep reinforcement learning, multi-agent system, transfer learning, curriculum learning, co-evolutionary multi-agent reinforcement learning, StarCraft II, SMAC
Subject Categories
Systems Engineering | Systems Science
Copyright
© Ayesha Siddiqua
Recommended Citation
Siddiqua, Ayesha, "Co-Evolving Multi-Agent Transfer Reinforcement Learning via Scenario Independent Representation" (2024). MSU Graduate Theses/Dissertations. 4017.
https://bearworks.missouristate.edu/theses/4017
