Cooperative pursuit of unauthorized UAVs in urban airspace via Multi-agent reinforcement learning

Abstract Urban Air Mobility (UAM) is an emergent concept for future air transportation. With UAM, cargo and passengers will be transported on-demand in urban airspace. UAM has shown a promising prospect in mitigating ground congestion and providing people with an alternative mobility option. However, unauthorized unmanned aerial vehicles (UAVs) in urban airspace present a significant threat to safety of UAM, drawing significant attention from research communities recently. Among all solutions, cooperative pursuit using a team of UAVs is an effective countermeasure for unauthorized UAVs in urban airspace. In this paper, we model cooperative pursuit as a pursuit-evasion game problem (PEG) and propose a multi-agent reinforcement learning (MARL) based approach to solve the problem efficiently. The proposed approach incorporates novel cellular-enabled parameter sharing and curriculum learning schemes to enhance the capability of pursuer UAVs in capturing faster unauthorized UAVs in urban airspace. Extensive experiments have been conducted using simulated urban airspace in order to evaluate the performance of the proposed method. Experimental results demonstrate that by incorporating the parameter sharing scheme, the proposed methods provide much higher capturing rates in a shorter time. Such superiority is more evident when communication constraints are more stringent and/or unauthorized UAVs are faster.