Date of Graduation

Summer 2021


Master of Natural and Applied Science in Computer Science


Computer Science

Committee Chair

Siming Liu


deep reinforcement learning, A2C, DenseNet, multi-agent system, influence map, StarCraft II, SMAC, MAIRL, MAIDRL

Subject Categories

Artificial Intelligence and Robotics


In recent years, reinforcement learning algorithms, a subset of machine learning that focuses on solving problems through trial-and-error learning, have been used in the field of multi-agent systems to help the agents with interactions and cooperation on a variety of tasks. Given the enormous success of reinforcement learning in single-agent systems like Chess, Shogi, and Go, it is natural for the next step to be the expansion into multi-agent systems. However, controlling multiple agents simultaneously is extremely challenging, as the complexity increases tremendously with the number of agents in the system. Existing approaches in this regard use a wide range of centralized, decentralized, semi-centralized, and even hybrid centralized-decentralized state representation methodologies. In this thesis, I propose a novel semi-centralized deep reinforcement learning algorithm, MAIDRL, for multi-agent interaction in mixed cooperative and competitive multi-agent environments. Specifically, I design a robust DenseNet-style actor-critic structured deep neural network for controlling multiple agents based upon the combination of local observations and abstracted global information to compete with opponent agents. I extract common knowledge through influence maps considering both enemy and friendly agents for fine-grained unit positioning and decision-making in combat. Compared to the centralized method, my design promotes a thorough understanding of the potential influence that a unit has without the need for a complete view of the global state. In addition, this design enables multi-agent understanding of a subset of the global information relative to common goals, unlike fully decentralized methods. The proposed method has been evaluated on StarCraft Multi-Agent Challenge scenarios in a real-time strategy game, StarCraft II, and the results show that, statistically, the agents controlled by MAIDRL perform better than or as competitive as those controlled by centralized and decentralized methods.


© Anthony Lee Harris

Open Access