Date of Graduation

Summer 2021

Degree

Master of Natural and Applied Science in Computer Science

Department

Computer Science

Committee Chair

Siming Liu

Keywords

deep reinforcement learning, A2C, DenseNet, multi-agent system, influence map, StarCraft II, SMAC, MAIRL, MAIDRL

Subject Categories

Artificial Intelligence and Robotics

Abstract

In recent years, reinforcement learning algorithms, a subset of machine learning that focuses on solving problems through trial-and-error learning, have been used in the field of multi-agent systems to help the agents with interactions and cooperation on a variety of tasks. Given the enormous success of reinforcement learning in single-agent systems like Chess, Shogi, and Go, it is natural for the next step to be the expansion into multi-agent systems. However, controlling multiple agents simultaneously is extremely challenging, as the complexity increases tremendously with the number of agents in the system. Existing approaches in this regard use a wide range of centralized, decentralized, semi-centralized, and even hybrid centralized-decentralized state representation methodologies. In this thesis, I propose a novel semi-centralized deep reinforcement learning algorithm, MAIDRL, for multi-agent interaction in mixed cooperative and competitive multi-agent environments. Specifically, I design a robust DenseNet-style actor-critic structured deep neural network for controlling multiple agents based upon the combination of local observations and abstracted global information to compete with opponent agents. I extract common knowledge through influence maps considering both enemy and friendly agents for fine-grained unit positioning and decision-making in combat. Compared to the centralized method, my design promotes a thorough understanding of the potential influence that a unit has without the need for a complete view of the global state. In addition, this design enables multi-agent understanding of a subset of the global information relative to common goals, unlike fully decentralized methods. The proposed method has been evaluated on StarCraft Multi-Agent Challenge scenarios in a real-time strategy game, StarCraft II, and the results show that, statistically, the agents controlled by MAIDRL perform better than or as competitive as those controlled by centralized and decentralized methods.

Copyright

© Anthony Lee Harris

Open Access

Share

COinS